SetFit with google-t5/t5-small

This is a SetFit model that can be used for Text Classification. This SetFit model uses google-t5/t5-small as the Sentence Transformer embedding model. A LogisticRegression instance is used for classification.

The model has been trained using an efficient few-shot learning technique that involves:

Fine-tuning a Sentence Transformer with contrastive learning.
Training a classification head with features from the fine-tuned Sentence Transformer.

Model Details

Model Description

Model Type: SetFit
Sentence Transformer body: google-t5/t5-small
Classification head: a LogisticRegression instance
Maximum Sequence Length: None tokens
Number of Classes: 5 classes

Model Sources

Repository: SetFit on GitHub
Paper: Efficient Few-Shot Learning Without Prompts
Blogpost: SetFit: Efficient Few-Shot Learning Without Prompts

Model Labels

Label	Examples
Tech Support	"My loyalty card isn't working at the checkout. What should I do?" 'How can I reset my password for the online account?' 'How can I reset my password for the online account?'
HR	"I'm interested in applying for a job at your company. Can you provide information on current openings?" 'I have a question about my paycheck. Who should I contact?' "I'm having an issue with my timesheet submission. Who should I contact?"
Product	'What brand of nut butters do you carry that are peanut-free?' 'Do you offer any delivery or pickup options for online grocery orders?' 'I have a dietary restriction - how can I easily identify suitable products?'
Returns	'My grocery delivery contained items that were spoiled or past their expiration date. How do I get replacements?' "I purchased a product that was supposed to be on sale but I didn't get the discounted price. Can I get a credit for the difference?" "I bought an item that doesn't fit. What's the process for exchanging it?"
Logistics	'My delivery was marked as "undeliverable" - what are the next steps I should take?' 'I need to change the delivery address for my upcoming order. How can I do that?' 'Is there a way to get real-time updates on the status of my order during the shipping process?'

Uses

Direct Use for Inference

First install the SetFit library:

pip install setfit

Then you can load this model and run inference.

from setfit import SetFitModel

# Download from the 🤗 Hub
model = SetFitModel.from_pretrained("setfit_model_id")
# Run inference
preds = model("Do you have any special deals or discounts on bulk items?")

Training Details

Training Set Metrics

Training set	Min	Median	Max
Word count	10	14.25	26

Label	Training Sample Count
Returns	8
Tech Support	8
Logistics	8
HR	8
Product	8

Training Hyperparameters

batch_size: (32, 32)
num_epochs: (100, 100)
max_steps: -1
sampling_strategy: oversampling
body_learning_rate: (2e-05, 1e-05)
head_learning_rate: 0.01
loss: CosineSimilarityLoss
distance_metric: cosine_distance
margin: 0.25
end_to_end: False
use_amp: False
warmup_proportion: 0.1
seed: 42
eval_max_steps: -1
load_best_model_at_end: False

Training Results

Epoch	Step	Training Loss	Validation Loss
0.025	1	0.2674	-
1.25	50	0.2345	-
2.5	100	0.2558	-
3.75	150	0.2126	-
5.0	200	0.1904	-
6.25	250	0.1965	-
7.5	300	0.2013	-
8.75	350	0.1221	-
10.0	400	0.1254	-
11.25	450	0.0791	-
12.5	500	0.0917	-
13.75	550	0.0757	-
15.0	600	0.0446	-
16.25	650	0.0407	-
17.5	700	0.0276	-
18.75	750	0.0297	-
20.0	800	0.017	-
21.25	850	0.0193	-
22.5	900	0.0105	-
23.75	950	0.0143	-
25.0	1000	0.0133	-
26.25	1050	0.0127	-
27.5	1100	0.0064	-
28.75	1150	0.0076	-
30.0	1200	0.0099	-
31.25	1250	0.0077	-
32.5	1300	0.0059	-
33.75	1350	0.0047	-
35.0	1400	0.0059	-
36.25	1450	0.005	-
37.5	1500	0.005	-
38.75	1550	0.005	-
40.0	1600	0.0043	-
41.25	1650	0.0056	-
42.5	1700	0.0036	-
43.75	1750	0.0029	-
45.0	1800	0.0031	-
46.25	1850	0.0033	-
47.5	1900	0.0028	-
48.75	1950	0.0042	-
50.0	2000	0.0038	-
51.25	2050	0.0032	-
52.5	2100	0.0033	-
53.75	2150	0.0031	-
55.0	2200	0.0023	-
56.25	2250	0.002	-
57.5	2300	0.003	-
58.75	2350	0.0039	-
60.0	2400	0.003	-
61.25	2450	0.0035	-
62.5	2500	0.0022	-
63.75	2550	0.0029	-
65.0	2600	0.0029	-
66.25	2650	0.0019	-
67.5	2700	0.002	-
68.75	2750	0.0041	-
70.0	2800	0.0022	-
71.25	2850	0.0027	-
72.5	2900	0.0016	-
73.75	2950	0.002	-
75.0	3000	0.0029	-
76.25	3050	0.0024	-
77.5	3100	0.0017	-
78.75	3150	0.0017	-
80.0	3200	0.0025	-
81.25	3250	0.0023	-
82.5	3300	0.0018	-
83.75	3350	0.0021	-
85.0	3400	0.0016	-
86.25	3450	0.0021	-
87.5	3500	0.0018	-
88.75	3550	0.0014	-
90.0	3600	0.0014	-
91.25	3650	0.0026	-
92.5	3700	0.0012	-
93.75	3750	0.0031	-
95.0	3800	0.0025	-
96.25	3850	0.0014	-
97.5	3900	0.0012	-
98.75	3950	0.0025	-
100.0	4000	0.002	-

Framework Versions

Python: 3.11.8
SetFit: 1.0.3
Sentence Transformers: 2.7.0
Transformers: 4.40.0
PyTorch: 2.2.2
Datasets: 2.19.0
Tokenizers: 0.19.1

Citation

BibTeX

@article{https://doi.org/10.48550/arxiv.2209.11055,
    doi = {10.48550/ARXIV.2209.11055},
    url = {https://arxiv.org/abs/2209.11055},
    author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
    keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
    title = {Efficient Few-Shot Learning Without Prompts},
    publisher = {arXiv},
    year = {2022},
    copyright = {Creative Commons Attribution 4.0 International}
}

wikd
/

t5-small-finetuned