Hugging Face's logo

twright8
/
setfit_lobbying_classifier

Text Classification
setfit
Safetensors
sentence-transformers
bert
generated_from_setfit_trainer
Eval Results
Model card Files Community
Edit model card

SetFit with BAAI/bge-base-en-v1.5

This is a SetFit model that can be used for Text Classification. This SetFit model uses BAAI/bge-base-en-v1.5 as the Sentence Transformer embedding model. A SetFitHead instance is used for classification.

The model has been trained using an efficient few-shot learning technique that involves:

  1. Fine-tuning a Sentence Transformer with contrastive learning.
  2. Training a classification head with features from the fine-tuned Sentence Transformer.

undefined = Health 1 = Housing 2 = Defence 3 = Climate

Model Details

Model Description

Model Sources

Evaluation

Metrics

Label F1 Accuracy
all 0.9667 0.9421

Uses

Direct Use for Inference

First install the SetFit library:

pip install setfit

Then you can load this model and run inference.

from setfit import SetFitModel

# Download from the 🤗 Hub
model = SetFitModel.from_pretrained("twright8/setfit_lobbying_classifier")
# Run inference
preds = model("Growth")

Training Details

Training Set Metrics

Training set Min Median Max
Word count 1 39.4538 282

Training Hyperparameters

  • batch_size: (16, 2)
  • num_epochs: (4, 9)
  • max_steps: -1
  • sampling_strategy: undersampling
  • body_learning_rate: (1.0797496673911536e-05, 3.457046714445997e-05)
  • head_learning_rate: 0.0004470582121407239
  • loss: CoSENTLoss
  • distance_metric: cosine_distance
  • margin: 0.25
  • end_to_end: True
  • use_amp: False
  • warmup_proportion: 0.1
  • seed: 42
  • eval_max_steps: -1
  • load_best_model_at_end: True

Training Results

Epoch Step Training Loss Validation Loss
0.0002 1 2.097 -
0.0077 50 8.5514 -
0.0155 100 3.5635 -
0.0232 150 2.9266 -
0.0310 200 2.1173 -
0.0387 250 3.1002 -
0.0465 300 3.6942 -
0.0542 350 3.4905 -
0.0620 400 4.0804 -
0.0697 450 1.6071 -
0.0774 500 2.3018 -
0.0852 550 2.3876 -
0.0929 600 0.2511 -
0.1007 650 0.2435 -
0.1084 700 2.2596 -
0.1162 750 1.121 -
0.1239 800 0.0907 -
0.1317 850 0.2172 -
0.1394 900 3.06 -
0.1471 950 0.0074 -
0.1549 1000 0.457 -
0.1626 1050 0.0575 -
0.1704 1100 0.0002 -
0.1781 1150 0.0003 -
0.1859 1200 0.0047 -
0.1936 1250 0.0004 -
0.2014 1300 0.0006 -
0.2091 1350 0.0027 -
0.2169 1400 0.0004 -
0.2246 1450 0.0009 -
0.2323 1500 0.0006 -
0.2401 1550 0.0003 -
0.2478 1600 0.0077 -
0.2556 1650 0.0004 -
0.2633 1700 0.0003 -
0.2711 1750 0.0005 -
0.2788 1800 0.0004 -
0.2866 1850 0.0007 -
0.2943 1900 0.0009 -
0.3020 1950 0.0062 -
0.3098 2000 0.0003 -
0.3175 2050 0.0001 -
0.3253 2100 0.0685 -
0.3330 2150 0.0008 -
0.3408 2200 0.0 -
0.3485 2250 0.0004 -
0.3563 2300 0.0004 -
0.3640 2350 0.0002 -
0.3717 2400 0.0001 -
0.3795 2450 0.0004 -
0.3872 2500 0.0004 -
0.3950 2550 0.0001 -
0.4027 2600 0.0001 -
0.4105 2650 0.0001 -
0.4182 2700 0.0005 -
0.4260 2750 0.0002 -
0.4337 2800 0.0001 -
0.4414 2850 0.0003 -
0.4492 2900 0.0005 -
0.4569 2950 0.0014 -
0.4647 3000 0.0001 -
0.4724 3050 0.0001 -
0.4802 3100 0.0002 -
0.4879 3150 0.0 -
0.4957 3200 0.0006 -
0.5034 3250 0.0 -
0.5112 3300 0.0 -
0.5189 3350 0.0002 -
0.5266 3400 0.0001 -
0.5344 3450 0.0006 -
0.5421 3500 0.0002 -
0.5499 3550 0.0001 -
0.5576 3600 0.0001 -
0.5654 3650 0.0001 -
0.5731 3700 0.0 -
0.5809 3750 0.0002 -
0.5886 3800 0.0 -
0.5963 3850 0.0044 -
0.6041 3900 0.0002 -
0.6118 3950 0.0001 -
0.6196 4000 0.0003 -
0.6273 4050 0.0005 -
0.6351 4100 0.0002 -
0.6428 4150 0.0 -
0.6506 4200 0.0003 -
0.6583 4250 0.0 -
0.6660 4300 0.0001 -
0.6738 4350 0.0 -
0.6815 4400 0.0008 -
0.6893 4450 0.0 -
0.6970 4500 0.0004 -
0.7048 4550 0.0001 -
0.7125 4600 0.0 -
0.7203 4650 0.0 -
0.7280 4700 0.0 -
0.7357 4750 0.0001 -
0.7435 4800 0.0001 -
0.7512 4850 0.001 -
0.7590 4900 0.0001 -
0.7667 4950 0.0 -
0.7745 5000 0.0001 -
0.7822 5050 0.0 -
0.7900 5100 0.0018 -
0.7977 5150 0.0001 -
0.8055 5200 0.0 -
0.8132 5250 0.0003 -
0.8209 5300 0.0003 -
0.8287 5350 0.0003 -
0.8364 5400 0.0001 -
0.8442 5450 0.0001 -
0.8519 5500 0.0001 -
0.8597 5550 0.0001 -
0.8674 5600 0.0001 -
0.8752 5650 0.0 -
0.8829 5700 0.0003 -
0.8906 5750 0.0003 -
0.8984 5800 0.0001 -
0.9061 5850 0.0001 -
0.9139 5900 0.0002 -
0.9216 5950 0.0 -
0.9294 6000 0.0001 -
0.9371 6050 0.0 -
0.9449 6100 0.0 -
0.9526 6150 0.0001 -
0.9603 6200 0.0 -
0.9681 6250 0.0001 -
0.9758 6300 0.0002 -
0.9836 6350 0.0 -
0.9913 6400 0.0 -
0.9991 6450 0.0002 -
1.0 6456 - 1.3837
1.0068 6500 0.0001 -
1.0146 6550 0.0001 -
1.0223 6600 0.0002 -
1.0300 6650 0.0001 -
1.0378 6700 0.0005 -
1.0455 6750 0.0001 -
1.0533 6800 0.0001 -
1.0610 6850 0.0 -
1.0688 6900 0.0 -
1.0765 6950 0.0009 -
1.0843 7000 0.0 -
1.0920 7050 0.0032 -
1.0998 7100 0.0001 -
1.1075 7150 0.0001 -
1.1152 7200 0.0001 -
1.1230 7250 0.0 -
1.1307 7300 0.0001 -
1.1385 7350 0.0 -
1.1462 7400 0.0 -
1.1540 7450 0.0002 -
1.1617 7500 0.0 -
1.1695 7550 0.0427 -
1.1772 7600 0.0 -
1.1849 7650 0.0 -
1.1927 7700 0.0 -
1.2004 7750 0.0002 -
1.2082 7800 0.0 -
1.2159 7850 0.0 -
1.2237 7900 0.0 -
1.2314 7950 0.0 -
1.2392 8000 0.0001 -
1.2469 8050 0.0 -
1.2546 8100 0.0001 -
1.2624 8150 0.0 -
1.2701 8200 0.0 -
1.2779 8250 0.0 -
1.2856 8300 0.0 -
1.2934 8350 0.0 -
1.3011 8400 0.0 -
1.3089 8450 0.0 -
1.3166 8500 0.0 -
1.3243 8550 0.0001 -
1.3321 8600 0.0 -
1.3398 8650 0.0002 -
1.3476 8700 0.0 -
1.3553 8750 0.0006 -
1.3631 8800 0.0 -
1.3708 8850 0.0 -
1.3786 8900 0.0001 -
1.3863 8950 0.0 -
1.3941 9000 0.0001 -
1.4018 9050 0.0 -
1.4095 9100 0.0002 -
1.4173 9150 0.0 -
1.4250 9200 0.0 -
1.4328 9250 0.0 -
1.4405 9300 0.0 -
1.4483 9350 0.0 -
1.4560 9400 0.0 -
1.4638 9450 0.0 -
1.4715 9500 0.0 -
1.4792 9550 0.0 -
1.4870 9600 0.0 -
1.4947 9650 0.0005 -
1.5025 9700 0.0 -
1.5102 9750 0.0001 -
1.5180 9800 0.0001 -
1.5257 9850 0.0001 -
1.5335 9900 0.0 -
1.5412 9950 0.0 -
1.5489 10000 0.0 -
1.5567 10050 0.0 -
1.5644 10100 0.0001 -
1.5722 10150 0.0 -
1.5799 10200 0.0002 -
1.5877 10250 0.0001 -
1.5954 10300 0.0005 -
1.6032 10350 0.0 -
1.6109 10400 0.0 -
1.6186 10450 0.0003 -
1.6264 10500 0.0002 -
1.6341 10550 0.0 -
1.6419 10600 0.0 -
1.6496 10650 0.0001 -
1.6574 10700 0.0002 -
1.6651 10750 0.0002 -
1.6729 10800 0.0054 -
1.6806 10850 0.0005 -
1.6884 10900 0.0001 -
1.6961 10950 0.0 -
1.7038 11000 0.0 -
1.7116 11050 0.0001 -
1.7193 11100 0.0001 -
1.7271 11150 0.0 -
1.7348 11200 0.0001 -
1.7426 11250 0.0 -
1.7503 11300 0.0001 -
1.7581 11350 0.0004 -
1.7658 11400 0.0 -
1.7735 11450 0.0001 -
1.7813 11500 0.0 -
1.7890 11550 0.0 -
1.7968 11600 0.0 -
1.8045 11650 0.0 -
1.8123 11700 0.0001 -
1.8200 11750 0.0002 -
1.8278 11800 0.0 -
1.8355 11850 0.0001 -
1.8432 11900 0.0 -
1.8510 11950 0.0001 -
1.8587 12000 0.0 -
1.8665 12050 0.0 -
1.8742 12100 0.0 -
1.8820 12150 0.0001 -
1.8897 12200 0.0 -
1.8975 12250 0.0 -
1.9052 12300 0.0 -
1.9129 12350 0.0 -
1.9207 12400 0.0 -
1.9284 12450 0.0 -
1.9362 12500 0.0 -
1.9439 12550 0.0003 -
1.9517 12600 0.0001 -
1.9594 12650 0.0 -
1.9672 12700 0.0001 -
1.9749 12750 0.0 -
1.9827 12800 0.0 -
1.9904 12850 0.0 -
1.9981 12900 0.0001 -
2.0 12912 - 2.611
2.0059 12950 0.0 -
2.0136 13000 0.0001 -
2.0214 13050 0.0001 -
2.0291 13100 0.0 -
2.0369 13150 0.0 -
2.0446 13200 0.0001 -
2.0524 13250 0.0 -
2.0601 13300 0.0002 -
2.0678 13350 0.0 -
2.0756 13400 0.0 -
2.0833 13450 0.0001 -
2.0911 13500 0.0001 -
2.0988 13550 0.0003 -
2.1066 13600 0.0 -
2.1143 13650 0.0001 -
2.1221 13700 0.0001 -
2.1298 13750 0.0001 -
2.1375 13800 0.0001 -
2.1453 13850 0.0 -
2.1530 13900 0.0 -
2.1608 13950 0.0 -
2.1685 14000 0.0 -
2.1763 14050 0.0 -
2.1840 14100 0.0001 -
2.1918 14150 0.0 -
2.1995 14200 0.0 -
2.2072 14250 0.0001 -
2.2150 14300 0.0 -
2.2227 14350 0.0 -
2.2305 14400 0.0004 -
2.2382 14450 0.0001 -
2.2460 14500 0.0 -
2.2537 14550 0.0003 -
2.2615 14600 0.0 -
2.2692 14650 0.0001 -
2.2770 14700 0.0001 -
2.2847 14750 0.0 -
2.2924 14800 0.0 -
2.3002 14850 0.0005 -
2.3079 14900 0.0 -
2.3157 14950 0.0002 -
2.3234 15000 0.0 -
2.3312 15050 0.0 -
2.3389 15100 0.0001 -
2.3467 15150 0.0001 -
2.3544 15200 0.0002 -
2.3621 15250 0.0001 -
2.3699 15300 0.0 -
2.3776 15350 0.0 -
2.3854 15400 0.0002 -
2.3931 15450 0.0003 -
2.4009 15500 0.0 -
2.4086 15550 0.0 -
2.4164 15600 0.0 -
2.4241 15650 0.0001 -
2.4318 15700 0.0 -
2.4396 15750 0.0 -
2.4473 15800 0.0002 -
2.4551 15850 0.0 -
2.4628 15900 0.0 -
2.4706 15950 0.0 -
2.4783 16000 0.0 -
2.4861 16050 0.0001 -
2.4938 16100 0.0 -
2.5015 16150 0.0 -
2.5093 16200 0.0 -
2.5170 16250 0.0 -
2.5248 16300 0.0 -
2.5325 16350 0.0 -
2.5403 16400 0.0 -
2.5480 16450 0.0 -
2.5558 16500 0.0 -
2.5635 16550 0.0001 -
2.5713 16600 0.0 -
2.5790 16650 0.0 -
2.5867 16700 0.0 -
2.5945 16750 0.0 -
2.6022 16800 0.0009 -
2.6100 16850 0.0001 -
2.6177 16900 0.0 -
2.6255 16950 0.0001 -
2.6332 17000 0.0 -
2.6410 17050 0.0 -
2.6487 17100 0.0001 -
2.6564 17150 0.0 -
2.6642 17200 0.0 -
2.6719 17250 0.0 -
2.6797 17300 0.0 -
2.6874 17350 0.0004 -
2.6952 17400 0.0 -
2.7029 17450 0.0 -
2.7107 17500 0.0 -
2.7184 17550 0.0 -
2.7261 17600 0.0 -
2.7339 17650 0.0 -
2.7416 17700 0.0001 -
2.7494 17750 0.0 -
2.7571 17800 0.0 -
2.7649 17850 0.0001 -
2.7726 17900 0.0 -
2.7804 17950 0.0001 -
2.7881 18000 0.0001 -
2.7958 18050 0.0 -
2.8036 18100 0.0 -
2.8113 18150 0.0 -
2.8191 18200 0.0 -
2.8268 18250 0.0 -
2.8346 18300 0.0001 -
2.8423 18350 0.0 -
2.8501 18400 0.0 -
2.8578 18450 0.0 -
2.8656 18500 0.0 -
2.8733 18550 0.0 -
2.8810 18600 0.0 -
2.8888 18650 0.0 -
2.8965 18700 0.0 -
2.9043 18750 0.0 -
2.9120 18800 0.0001 -
2.9198 18850 0.0 -
2.9275 18900 0.0 -
2.9353 18950 0.0 -
2.9430 19000 0.0 -
2.9507 19050 0.0 -
2.9585 19100 0.0 -
2.9662 19150 0.0 -
2.9740 19200 0.0 -
2.9817 19250 0.0003 -
2.9895 19300 0.0001 -
2.9972 19350 0.0 -
3.0 19368 - 2.0845
3.0050 19400 0.0 -
3.0127 19450 0.0001 -
3.0204 19500 0.0 -
3.0282 19550 0.0 -
3.0359 19600 0.0 -
3.0437 19650 0.0 -
3.0514 19700 0.0 -
3.0592 19750 0.0 -
3.0669 19800 0.0001 -
3.0747 19850 0.0 -
3.0824 19900 0.0 -
3.0901 19950 0.0001 -
3.0979 20000 0.0 -
3.1056 20050 0.0 -
3.1134 20100 0.0 -
3.1211 20150 0.0001 -
3.1289 20200 0.0 -
3.1366 20250 0.0 -
3.1444 20300 0.0 -
3.1521 20350 0.0 -
3.1599 20400 0.0 -
3.1676 20450 0.0001 -
3.1753 20500 0.0 -
3.1831 20550 0.0001 -
3.1908 20600 0.0 -
3.1986 20650 0.0 -
3.2063 20700 0.0 -
3.2141 20750 0.0 -
3.2218 20800 0.0 -
3.2296 20850 0.0003 -
3.2373 20900 0.0 -
3.2450 20950 0.0 -
3.2528 21000 0.0 -
3.2605 21050 0.0 -
3.2683 21100 0.0001 -
3.2760 21150 0.0001 -
3.2838 21200 0.0 -
3.2915 21250 0.0 -
3.2993 21300 0.0 -
3.3070 21350 0.0 -
3.3147 21400 0.0 -
3.3225 21450 0.0001 -
3.3302 21500 0.0 -
3.3380 21550 0.0 -
3.3457 21600 0.0 -
3.3535 21650 0.0 -
3.3612 21700 0.0 -
3.3690 21750 0.0 -
3.3767 21800 0.0 -
3.3844 21850 0.0 -
3.3922 21900 0.0001 -
3.3999 21950 0.0 -
3.4077 22000 0.0 -
3.4154 22050 0.0001 -
3.4232 22100 0.0 -
3.4309 22150 0.0001 -
3.4387 22200 0.0 -
3.4464 22250 0.0 -
3.4542 22300 0.0 -
3.4619 22350 0.0001 -
3.4696 22400 0.0 -
3.4774 22450 0.0 -
3.4851 22500 0.0 -
3.4929 22550 0.0001 -
3.5006 22600 0.0002 -
3.5084 22650 0.0001 -
3.5161 22700 0.0 -
3.5239 22750 0.0001 -
3.5316 22800 0.0 -
3.5393 22850 0.0 -
3.5471 22900 0.0001 -
3.5548 22950 0.0 -
3.5626 23000 0.0 -
3.5703 23050 0.0 -
3.5781 23100 0.0 -
3.5858 23150 0.0001 -
3.5936 23200 0.0 -
3.6013 23250 0.0001 -
3.6090 23300 0.0001 -
3.6168 23350 0.0 -
3.6245 23400 0.0003 -
3.6323 23450 0.0 -
3.6400 23500 0.0 -
3.6478 23550 0.0001 -
3.6555 23600 0.0 -
3.6633 23650 0.0 -
3.6710 23700 0.0 -
3.6787 23750 0.0001 -
3.6865 23800 0.0 -
3.6942 23850 0.0001 -
3.7020 23900 0.0002 -
3.7097 23950 0.0 -
3.7175 24000 0.0 -
3.7252 24050 0.0 -
3.7330 24100 0.0 -
3.7407 24150 0.0001 -
3.7485 24200 0.0 -
3.7562 24250 0.0 -
3.7639 24300 0.0 -
3.7717 24350 0.0 -
3.7794 24400 0.0 -
3.7872 24450 0.0 -
3.7949 24500 0.0001 -
3.8027 24550 0.0001 -
3.8104 24600 0.0 -
3.8182 24650 0.0 -
3.8259 24700 0.0 -
3.8336 24750 0.0 -
3.8414 24800 0.0001 -
3.8491 24850 0.0 -
3.8569 24900 0.0 -
3.8646 24950 0.0 -
3.8724 25000 0.0 -
3.8801 25050 0.0 -
3.8879 25100 0.0 -
3.8956 25150 0.0001 -
3.9033 25200 0.0 -
3.9111 25250 0.0002 -
3.9188 25300 0.0001 -
3.9266 25350 0.0 -
3.9343 25400 0.0 -
3.9421 25450 0.0 -
3.9498 25500 0.0001 -
3.9576 25550 0.0 -
3.9653 25600 0.0 -
3.9730 25650 0.0001 -
3.9808 25700 0.0 -
3.9885 25750 0.0 -
3.9963 25800 0.0 -
4.0 25824 - 2.3576
  • The bold row denotes the saved checkpoint.

Framework Versions

  • Python: 3.10.12
  • SetFit: 1.0.3
  • Sentence Transformers: 3.0.1
  • Transformers: 4.39.0
  • PyTorch: 2.3.0+cu121
  • Datasets: 2.20.0
  • Tokenizers: 0.15.2

Citation

BibTeX 

@article{https://doi.org/10.48550/arxiv.2209.11055,
    doi = {10.48550/ARXIV.2209.11055},
    url = {https://arxiv.org/abs/2209.11055},
    author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
    keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
    title = {Efficient Few-Shot Learning Without Prompts},
    publisher = {arXiv},
    year = {2022},
    copyright = {Creative Commons Attribution 4.0 International}
}
Downloads last month
2
Safetensors
Model size
109M params
Tensor type
F32
·
Inference Examples
Text Classification
Inference API (serverless) is not available, repository is disabled.

Model tree for twright8/setfit_lobbying_classifier

Base model

BAAI/bge-base-en-v1.5
Finetuned
this model

Evaluation results

View on Papers With Code