SetFit with sentence-transformers/paraphrase-mpnet-base-v2

This is a SetFit model that can be used for Text Classification. This SetFit model uses sentence-transformers/paraphrase-mpnet-base-v2 as the Sentence Transformer embedding model. A LogisticRegression instance is used for classification.

The model has been trained using an efficient few-shot learning technique that involves:

Fine-tuning a Sentence Transformer with contrastive learning.
Training a classification head with features from the fine-tuned Sentence Transformer.

Model Details

Model Description

Model Type: SetFit
Sentence Transformer body: sentence-transformers/paraphrase-mpnet-base-v2
Classification head: a LogisticRegression instance
Maximum Sequence Length: 512 tokens
Number of Classes: 3 classes

Model Sources

Repository: SetFit on GitHub
Paper: Efficient Few-Shot Learning Without Prompts
Blogpost: SetFit: Efficient Few-Shot Learning Without Prompts

Model Labels

Label	Examples
neutral	'Administrative Resolution APS/DJ/DI/No. 221/2021 establishes that for fiscal year 2021 the maximum limit for investments by insurance companies set by the Central Bank of Bolivia includes investments in debt securities issued abroad by the National General Treasury (Bolivian Sovereign Bonds).' 'Decree No. 04 of the head of the Insurance State Supervision Service replaced Decree No. 51/01 of the president of the National Bank of Georgia as the regulation governing insurance companies and pension funds.' 'The Law on the Securities Market was promulgated, setting out the legal framework governing capital market operations.'
loosen	'Limits on transfers abroad were increased. Depositors may transfer within their normal business activity and on presentation of supporting documents up to €20,000 a day an account without approval and €20,001 to €300,000 a transaction with approval by the committee established by the MOF and the Central Bank of Cyprus based on the liquidity buffer situation of the bank. Amounts exceeding €300,000 a transaction require individual approval of the committee.' 'According to the Consolidated Law of Finance (Legislative Decree No. 58 of February 24, 1998) as amended by Legislative Decree No. 44/2014, if mutual funds are covered under EU directives (UCITS and European Directive on Alternative Investment Fund Managers), the CONSOB must be notified before the offering. Under the same amendment, offering securities issued by mutual funds that are not covered under EU directives is not allowed. Previously, offerings of securities issued by mutual funds not covered under EU directives were subject to authorization.' 'The capital outflow tax was reduced to 4.25% from 4.50%.'
tighten	'Bank Indonesia Regulation No. 14/25/PBI/2012 of December 27, 2012, concerning Receipt of Export Proceeds and Withdrawal of Foreign Currency on External Debt refines Bank Indonesia Regulations Nos. 13/20/PBI/2011 and 14/11/PBI/2012. The new regulation aims to ensure that receipt of foreign exchange proceeds from debt issuance abroad takes place through a domestic foreign exchange bank in the Indonesian banking system. These funds do not have to be kept in a domestic bank and they may be freely transferred abroad. The foreign exchange does not have to be converted to domestic currency. This requirement did not apply to agreements signed before January 2, 2012, during a transition period that ended December 31, 2012.' 'Investment Proclamation No. 1180/2020 introduced a new framework to register and administer foreign direct investment. It included provisions for a minimum capital amount per project a foreign investor must invest ranging from US$50,000 to US$200,000 depending on the type of investment being undertaken.' 'External debt of publicly owned enterprises must be approved by the Superior Strategic Council of Publicly Owned Enterprises (Consejo Superior Estratégico de la Empresa Pública—COSEEP) pursuant to Law No. 466 of December 27, 2013.'

Evaluation

Metrics

Label	Accuracy
all	0.8281

Uses

Direct Use for Inference

First install the SetFit library:

pip install setfit

Then you can load this model and run inference.

from setfit import SetFitModel

# Download from the 🤗 Hub
model = SetFitModel.from_pretrained("kteoh37/setfit_sectionxi_changes_100_examples")
# Run inference
preds = model("Constitutional changes prohibit selling agricultural land to foreigners.")

Training Details

Training Set Metrics

Training set	Min	Median	Max
Word count	6	38.4936	225

Label	Training Sample Count
loosen	100
tighten	100
neutral	35

Training Hyperparameters

batch_size: (16, 16)
num_epochs: (10, 10)
max_steps: -1
sampling_strategy: oversampling
body_learning_rate: (2e-05, 1e-05)
head_learning_rate: 0.01
loss: CosineSimilarityLoss
distance_metric: cosine_distance
margin: 0.25
end_to_end: False
use_amp: False
warmup_proportion: 0.1
seed: 42
eval_max_steps: -1
load_best_model_at_end: True

Training Results

Epoch	Step	Training Loss	Validation Loss
0.0005	1	0.2629	-
0.0235	50	0.2148	-
0.0471	100	0.2342	-
0.0706	150	0.2326	-
0.0941	200	0.2391	-
0.1176	250	0.2351	-
0.1412	300	0.1484	-
0.1647	350	0.1034	-
0.1882	400	0.123	-
0.2118	450	0.064	-
0.2353	500	0.0697	-
0.2588	550	0.0367	-
0.2824	600	0.0695	-
0.3059	650	0.0044	-
0.3294	700	0.009	-
0.3529	750	0.0027	-
0.3765	800	0.0012	-
0.4	850	0.0609	-
0.4235	900	0.0019	-
0.4471	950	0.0013	-
0.4706	1000	0.0031	-
0.4941	1050	0.0004	-
0.5176	1100	0.0029	-
0.5412	1150	0.001	-
0.5647	1200	0.0018	-
0.5882	1250	0.0023	-
0.6118	1300	0.0003	-
0.6353	1350	0.0007	-
0.6588	1400	0.0204	-
0.6824	1450	0.0486	-
0.7059	1500	0.0056	-
0.7294	1550	0.0003	-
0.7529	1600	0.0052	-
0.7765	1650	0.0033	-
0.8	1700	0.0021	-
0.8235	1750	0.0003	-
0.8471	1800	0.0111	-
0.8706	1850	0.0003	-
0.8941	1900	0.0531	-
0.9176	1950	0.0075	-
0.9412	2000	0.0003	-
0.9647	2050	0.0022	-
0.9882	2100	0.0006	-
1.0	2125	-	0.1384
1.0118	2150	0.0001	-
1.0353	2200	0.0001	-
1.0588	2250	0.0011	-
1.0824	2300	0.0001	-
1.1059	2350	0.0002	-
1.1294	2400	0.0	-
1.1529	2450	0.0002	-
1.1765	2500	0.0003	-
1.2	2550	0.0116	-
1.2235	2600	0.0002	-
1.2471	2650	0.0002	-
1.2706	2700	0.0001	-
1.2941	2750	0.0002	-
1.3176	2800	0.0	-
1.3412	2850	0.0132	-
1.3647	2900	0.0265	-
1.3882	2950	0.0035	-
1.4118	3000	0.0003	-
1.4353	3050	0.0022	-
1.4588	3100	0.0013	-
1.4824	3150	0.0006	-
1.5059	3200	0.0003	-
1.5294	3250	0.0	-
1.5529	3300	0.0198	-
1.5765	3350	0.0001	-
1.6	3400	0.0	-
1.6235	3450	0.0001	-
1.6471	3500	0.0	-
1.6706	3550	0.0	-
1.6941	3600	0.0002	-
1.7176	3650	0.0	-
1.7412	3700	0.0	-
1.7647	3750	0.0023	-
1.7882	3800	0.0	-
1.8118	3850	0.0074	-
1.8353	3900	0.0004	-
1.8588	3950	0.0001	-
1.8824	4000	0.0228	-
1.9059	4050	0.0256	-
1.9294	4100	0.0316	-
1.9529	4150	0.0001	-
1.9765	4200	0.0	-
2.0	4250	0.0	0.1386
2.0235	4300	0.0006	-
2.0471	4350	0.0001	-
2.0706	4400	0.0072	-
2.0941	4450	0.0433	-
2.1176	4500	0.0001	-
2.1412	4550	0.0004	-
2.1647	4600	0.0	-
2.1882	4650	0.0024	-
2.2118	4700	0.0	-
2.2353	4750	0.0001	-
2.2588	4800	0.0	-
2.2824	4850	0.0002	-
2.3059	4900	0.0001	-
2.3294	4950	0.0	-
2.3529	5000	0.002	-
2.3765	5050	0.0303	-
2.4	5100	0.0799	-
2.4235	5150	0.0001	-
2.4471	5200	0.0	-
2.4706	5250	0.0024	-
2.4941	5300	0.0001	-
2.5176	5350	0.0138	-
2.5412	5400	0.0	-
2.5647	5450	0.0001	-
2.5882	5500	0.0	-
2.6118	5550	0.0	-
2.6353	5600	0.0	-
2.6588	5650	0.0	-
2.6824	5700	0.0396	-
2.7059	5750	0.0001	-
2.7294	5800	0.0	-
2.7529	5850	0.0002	-
2.7765	5900	0.0001	-
2.8	5950	0.0037	-
2.8235	6000	0.0	-
2.8471	6050	0.0186	-
2.8706	6100	0.0043	-
2.8941	6150	0.0315	-
2.9176	6200	0.0144	-
2.9412	6250	0.0	-
2.9647	6300	0.0052	-
2.9882	6350	0.0003	-
3.0	6375	-	0.1526
3.0118	6400	0.0	-
3.0353	6450	0.0002	-
3.0588	6500	0.0011	-
3.0824	6550	0.0	-
3.1059	6600	0.0	-
3.1294	6650	0.0002	-
3.1529	6700	0.0001	-
3.1765	6750	0.0002	-
3.2	6800	0.0191	-
3.2235	6850	0.0001	-
3.2471	6900	0.0	-
3.2706	6950	0.0036	-
3.2941	7000	0.0001	-
3.3176	7050	0.0197	-
3.3412	7100	0.0101	-
3.3647	7150	0.0	-
3.3882	7200	0.0	-
3.4118	7250	0.0003	-
3.4353	7300	0.0001	-
3.4588	7350	0.0	-
3.4824	7400	0.0001	-
3.5059	7450	0.0174	-
3.5294	7500	0.0	-
3.5529	7550	0.0	-
3.5765	7600	0.0	-
3.6	7650	0.0	-
3.6235	7700	0.0012	-
3.6471	7750	0.0	-
3.6706	7800	0.0	-
3.6941	7850	0.0	-
3.7176	7900	0.0	-
3.7412	7950	0.0	-
3.7647	8000	0.0	-
3.7882	8050	0.0	-
3.8118	8100	0.0	-
3.8353	8150	0.0004	-
3.8588	8200	0.0	-
3.8824	8250	0.0154	-
3.9059	8300	0.0201	-
3.9294	8350	0.0332	-
3.9529	8400	0.0	-
3.9765	8450	0.0002	-
4.0	8500	0.0028	0.1434
4.0235	8550	0.0001	-
4.0471	8600	0.0	-
4.0706	8650	0.0077	-
4.0941	8700	0.0435	-
4.1176	8750	0.0	-
4.1412	8800	0.0001	-
4.1647	8850	0.0	-
4.1882	8900	0.0024	-
4.2118	8950	0.0	-
4.2353	9000	0.0	-
4.2588	9050	0.0	-
4.2824	9100	0.0002	-
4.3059	9150	0.0	-
4.3294	9200	0.0263	-
4.3529	9250	0.0	-
4.3765	9300	0.0	-
4.4	9350	0.0416	-
4.4235	9400	0.0	-
4.4471	9450	0.0061	-
4.4706	9500	0.0121	-
4.4941	9550	0.0001	-
4.5176	9600	0.0187	-
4.5412	9650	0.0	-
4.5647	9700	0.0	-
4.5882	9750	0.0	-
4.6118	9800	0.0	-
4.6353	9850	0.0	-
4.6588	9900	0.0117	-
4.6824	9950	0.0367	-
4.7059	10000	0.006	-
4.7294	10050	0.0	-
4.7529	10100	0.0002	-
4.7765	10150	0.0003	-
4.8	10200	0.0	-
4.8235	10250	0.0	-
4.8471	10300	0.0085	-
4.8706	10350	0.0	-
4.8941	10400	0.0369	-
4.9176	10450	0.0091	-
4.9412	10500	0.0	-
4.9647	10550	0.0	-
4.9882	10600	0.0	-
5.0	10625	-	0.1711
5.0118	10650	0.0	-
5.0353	10700	0.0	-
5.0588	10750	0.0	-
5.0824	10800	0.0238	-
5.1059	10850	0.0	-
5.1294	10900	0.0075	-
5.1529	10950	0.0	-
5.1765	11000	0.0	-
5.2	11050	0.0179	-
5.2235	11100	0.0	-
5.2471	11150	0.0	-
5.2706	11200	0.0171	-
5.2941	11250	0.0002	-
5.3176	11300	0.0	-
5.3412	11350	0.0128	-
5.3647	11400	0.0	-
5.3882	11450	0.0029	-
5.4118	11500	0.0	-
5.4353	11550	0.0	-
5.4588	11600	0.0	-
5.4824	11650	0.0	-
5.5059	11700	0.0	-
5.5294	11750	0.0	-
5.5529	11800	0.0	-
5.5765	11850	0.0001	-
5.6	11900	0.0001	-
5.6235	11950	0.0001	-
5.6471	12000	0.0	-
5.6706	12050	0.0001	-
5.6941	12100	0.0	-
5.7176	12150	0.0	-
5.7412	12200	0.0	-
5.7647	12250	0.0	-
5.7882	12300	0.006	-
5.8118	12350	0.0001	-
5.8353	12400	0.0042	-
5.8588	12450	0.0001	-
5.8824	12500	0.0	-
5.9059	12550	0.017	-
5.9294	12600	0.0282	-
5.9529	12650	0.0	-
5.9765	12700	0.0046	-
6.0	12750	0.0	0.1451
6.0235	12800	0.0001	-
6.0471	12850	0.0002	-
6.0706	12900	0.0625	-
6.0941	12950	0.0633	-
6.1176	13000	0.0598	-
6.1412	13050	0.0001	-
6.1647	13100	0.0012	-
6.1882	13150	0.0004	-
6.2118	13200	0.0	-
6.2353	13250	0.0002	-
6.2588	13300	0.0	-
6.2824	13350	0.0608	-
6.3059	13400	0.0006	-
6.3294	13450	0.0	-
6.3529	13500	0.0587	-
6.3765	13550	0.0003	-
6.4	13600	0.0429	-
6.4235	13650	0.0	-
6.4471	13700	0.0	-
6.4706	13750	0.0001	-
6.4941	13800	0.0	-
6.5176	13850	0.0135	-
6.5412	13900	0.019	-
6.5647	13950	0.0028	-
6.5882	14000	0.0	-
6.6118	14050	0.0	-
6.6353	14100	0.0169	-
6.6588	14150	0.0167	-
6.6824	14200	0.0375	-
6.7059	14250	0.0	-
6.7294	14300	0.0044	-
6.7529	14350	0.0	-
6.7765	14400	0.0	-
6.8	14450	0.0025	-
6.8235	14500	0.0033	-
6.8471	14550	0.0145	-
6.8706	14600	0.0	-
6.8941	14650	0.0346	-
6.9176	14700	0.0117	-
6.9412	14750	0.0001	-
6.9647	14800	0.0	-
6.9882	14850	0.0	-
7.0	14875	-	0.1828
7.0118	14900	0.0	-
7.0353	14950	0.0	-
7.0588	15000	0.0031	-
7.0824	15050	0.0001	-
7.1059	15100	0.0055	-
7.1294	15150	0.0208	-
7.1529	15200	0.0	-
7.1765	15250	0.0	-
7.2	15300	0.0134	-
7.2235	15350	0.0	-
7.2471	15400	0.0222	-
7.2706	15450	0.0	-
7.2941	15500	0.0001	-
7.3176	15550	0.0	-
7.3412	15600	0.0111	-
7.3647	15650	0.0	-
7.3882	15700	0.0025	-
7.4118	15750	0.0	-
7.4353	15800	0.0	-
7.4588	15850	0.0	-
7.4824	15900	0.0	-
7.5059	15950	0.0	-
7.5294	16000	0.0	-
7.5529	16050	0.0	-
7.5765	16100	0.0	-
7.6	16150	0.0	-
7.6235	16200	0.0001	-
7.6471	16250	0.0203	-
7.6706	16300	0.0	-
7.6941	16350	0.0	-
7.7176	16400	0.0	-
7.7412	16450	0.0186	-
7.7647	16500	0.0	-
7.7882	16550	0.0	-
7.8118	16600	0.0049	-
7.8353	16650	0.0	-
7.8588	16700	0.0044	-
7.8824	16750	0.0266	-
7.9059	16800	0.015	-
7.9294	16850	0.0331	-
7.9529	16900	0.0	-
7.9765	16950	0.0	-
8.0	17000	0.0	0.1778
8.0235	17050	0.0	-
8.0471	17100	0.0	-
8.0706	17150	0.0082	-
8.0941	17200	0.0414	-
8.1176	17250	0.0	-
8.1412	17300	0.0	-
8.1647	17350	0.0025	-
8.1882	17400	0.0	-
8.2118	17450	0.0	-
8.2353	17500	0.0	-
8.2588	17550	0.0	-
8.2824	17600	0.0	-
8.3059	17650	0.0	-
8.3294	17700	0.0033	-
8.3529	17750	0.0	-
8.3765	17800	0.0033	-
8.4	17850	0.0371	-
8.4235	17900	0.0217	-
8.4471	17950	0.004	-
8.4706	18000	0.0	-
8.4941	18050	0.0	-
8.5176	18100	0.0179	-
8.5412	18150	0.0	-
8.5647	18200	0.0	-
8.5882	18250	0.0032	-
8.6118	18300	0.0	-
8.6353	18350	0.0026	-
8.6588	18400	0.0	-
8.6824	18450	0.0387	-
8.7059	18500	0.0	-
8.7294	18550	0.0	-
8.7529	18600	0.0204	-
8.7765	18650	0.0212	-
8.8	18700	0.0	-
8.8235	18750	0.0065	-
8.8471	18800	0.0114	-
8.8706	18850	0.0	-
8.8941	18900	0.0335	-
8.9176	18950	0.0147	-
8.9412	19000	0.0	-
8.9647	19050	0.0053	-
8.9882	19100	0.0	-
9.0	19125	-	0.1773
9.0118	19150	0.0181	-
9.0353	19200	0.0	-
9.0588	19250	0.0	-
9.0824	19300	0.0	-
9.1059	19350	0.0226	-
9.1294	19400	0.0	-
9.1529	19450	0.0	-
9.1765	19500	0.0	-
9.2	19550	0.013	-
9.2235	19600	0.0036	-
9.2471	19650	0.0	-
9.2706	19700	0.0	-
9.2941	19750	0.0	-
9.3176	19800	0.0	-
9.3412	19850	0.0537	-
9.3647	19900	0.0	-
9.3882	19950	0.0031	-
9.4118	20000	0.0	-
9.4353	20050	0.0	-
9.4588	20100	0.0	-
9.4824	20150	0.0	-
9.5059	20200	0.0	-
9.5294	20250	0.0	-
9.5529	20300	0.0033	-
9.5765	20350	0.0	-
9.6	20400	0.0	-
9.6235	20450	0.0	-
9.6471	20500	0.0	-
9.6706	20550	0.0035	-
9.6941	20600	0.0	-
9.7176	20650	0.0036	-
9.7412	20700	0.0035	-
9.7647	20750	0.0	-
9.7882	20800	0.0	-
9.8118	20850	0.0	-
9.8353	20900	0.0	-
9.8588	20950	0.0	-
9.8824	21000	0.0036	-
9.9059	21050	0.0127	-
9.9294	21100	0.0364	-
9.9529	21150	0.0	-
9.9765	21200	0.0	-
10.0	21250	0.0	0.1803

The bold row denotes the saved checkpoint.

Framework Versions

Python: 3.10.12
SetFit: 1.0.3
Sentence Transformers: 3.1.0
Transformers: 4.39.0
PyTorch: 2.4.0+cu121
Datasets: 3.0.0
Tokenizers: 0.15.2

Citation

BibTeX

@article{https://doi.org/10.48550/arxiv.2209.11055,
    doi = {10.48550/ARXIV.2209.11055},
    url = {https://arxiv.org/abs/2209.11055},
    author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
    keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
    title = {Efficient Few-Shot Learning Without Prompts},
    publisher = {arXiv},
    year = {2022},
    copyright = {Creative Commons Attribution 4.0 International}
}

kteoh37
/

setfit_sectionxi_changes_100_examples