edwsiew/phantom-dispatch-02

SetFit with sentence-transformers/paraphrase-mpnet-base-v2

This is a SetFit model that can be used for Text Classification. This SetFit model uses sentence-transformers/paraphrase-mpnet-base-v2 as the Sentence Transformer embedding model. A LogisticRegression instance is used for classification.

The model has been trained using an efficient few-shot learning technique that involves:

1. Fine-tuning a Sentence Transformer with contrastive learning.
2. Training a classification head with features from the fine-tuned Sentence Transformer.

Model Details

Model Description

• Model Type: SetFit
• Sentence Transformer body: sentence-transformers/paraphrase-mpnet-base-v2
• Classification head: a LogisticRegression instance
• Maximum Sequence Length: 512 tokens
• Number of Classes: 2 classes

Model Sources

• Repository: SetFit on GitHub
• Paper: Efficient Few-Shot Learning Without Prompts
• Blogpost: SetFit: Efficient Few-Shot Learning Without Prompts

Model Labels

Label	Examples
1	'troubleshooting triage category generator shut down oss netcool alarms ccl04246 rbs generator shut down fieldreplaceableunit=sau alarmport=5 rbs generator shut down 2024 07 04 16 31 52 smart alarm ccl04246 rbs generator shut down fieldreplaceableunit=sau alarmport=5 2024 07 04 16 31 36 mdat oremis verification generac baldor magnum repeats open related tckt active eim intrusionknowledge judgement sending to vendor to investigate and resolve gen shut down condition dispatch strategy vendor test results triage category generator shut down oss netcool alarms ccl04246 rbs generator shut down fieldreplaceableunit=sau alarmport=5 rbs generator shut down 2024 07 04 16 31 52 smart alarm ccl04246 rbs generator shut down fieldreplaceableunit=sau alarmport=5 2024 07 04 16 31 36 mdat oremis verification generac baldor magnum repeats open related tckt active eim intrusionknowledge judgement sending to vendor to investigate and resolve gen shut down condition dispatch strategy vendor trouble description triage category generator shut down oss netcool alarms ccl04246 rbs generator shut down fieldreplaceableunit=sau alarmport=5 rbs generator shut down 2024 07 04 16 31 52 smart alarm ccl04246 rbs generator shut down fieldreplaceableunit=sau alarmport=5 2024 07 04 16 31 36 mdat oremis verification generac baldor magnum repeats open related tckt active eim intrusionknowledge judgement sending to vendor to investigate and resolve gen shut down condition dispatch strategy vendor history of trouble na vendor acas problem description triage category generator shut down oss netcool alarms ccl04246 rbs generator shut down fieldreplaceableunit=sau alarmport=5 rbs generator shut down 2024 07 04 16 31 52 smart alarm ccl04246 rbs generator shut down fieldreplaceableunit=sau alarmport=5 2024 07 04 16 31 36 mdat oremis verification generac baldor magnum repeats open related tckt active eim intrusionknowledge judgement sending to vendor to investigate and resolve gen shut down condition dispatch strategy vendor special access na' 'troubleshooting triage category rbs generator fuel leak alarm cvl08526 cvl08526 rbs generator fuel leak fieldreplaceableunit=sau 1 alarmport=23 2024 07 10 13 07 38 cvl08526 cvl08526 rbs rbs generator fuel leak fieldreplaceableunit=sau 1 alarmport=20 2024 07 10 13 05 04 mdat oremis verification generator generac baldor magnum sd30 manufacturer generac baldor magnum model sd30 status in use serial 3008406953 kw 30 prime power source no still on site yes engine perkins engine co ltd 404d 22ta manufacturer perkins engine co ltd model 404d 22ta serial gr84695u9967000g max engine kw 36 manufacturered date 2021 02 01 engine type diesel max brake hp 49 in service date 2022 07 13 fuel type ultra low sulfur diesel ulsd owner cell no repeats open related tckt active eim intrusion knowledge judgement sending to vendor to investigate and resolve gen rbs generator fuel leak condition dispatch strategy vendor test results triage category generator rbs generator fuel leak alarm cvl08526 cvl08526 rbs generator fuel leak fieldreplaceableunit=sau 1 alarmport=23 2024 07 10 13 07 38 cvl08526 cvl08526 rbs generator rbs generator fuel leak fieldreplaceableunit=sau 1 alarmport=20 2024 07 10 13 05 04 mdat oremis verification generator generac baldor magnum sd30 manufacturer generac baldor magnum model sd30 status in use serial 3008406953 kw 30 prime power source no still on site yes engine perkins engine co ltd 404d 22ta manufacturer perkins engine co ltd model 404d 22ta serial gr84695u9967000g max engine kw 36 manufacturered date 2021 02 01 engine type diesel max brake hp 49 in service date 2022 07 13 fuel type ultra low sulfur diesel ulsd owner cell no repeats open related tckt active eim intrusion knowledge judgement sending to vendor to investigate and resolve gen rbs generator fuel leak condition dispatch strategy vendor trouble description smart rbs generator fuel leak history of trouble na vendor acas problem description smart rbs generator fuel leak special access na' 'troubleshooting performed blackout test test results transfer switch display goes dark when commercial power is shut off generator does not start trouble description please create a work order for generator contractor to troubleshoot fixed generator transfer switch transfer switch failed blackout test and needs battery replaced will result in full site outage if commercial power is lost history of trouble unknown vendor acas problem description transfer switch display goes dark when commercial power is shut off generator does not start special access none'
0	'generaro will not start also please check the timer for tuesdays starting 9am alarm on the display oil pressure shutdown try to reset an start it but no luick poc davila rosalio requestor email rosalio davila verizonwireless com requestor phone 602 689 5506' 'troubleshooting triage category gen fail cli alarms rbs generator shut down fieldreplaceableunit=sau 1 alarmport=9 2024 08 28 19 24 42 mdat verification y generator generac baldor magnum sd30 3008240427 fixed knowledge judgement sending to vendor to check generator dispatch strategy vendor test results triage category gen fail cli alarms rbs generator shut down fieldreplaceableunit=sau 1 alarmport=9 2024 08 28 19 24 42 mdat verification y generator generac baldor magnum sd30 3008240427 fixed knowledge judgement sending to vendor to check generator dispatch strategy vendor trouble description triage category gen fail cli alarms rbs generator shut down fieldreplaceableunit=sau 1 alarmport=9 2024 08 28 19 24 42 mdat verification y generator generac baldor magnum sd30 3008240427 fixed knowledge judgement sending to vendor to check generator dispatch strategy vendor history of trouble n a special access n a vendor acas problem description triage category gen fail cli alarms rbs generator shut down fieldreplaceableunit=sau 1 alarmport=9 2024 08 28 19 24 42 mdat verification y generator generac baldor magnum sd30 3008240427 fixed knowledge judgement sending to vendor to check generator dispatch strategy vendor wo po 2024ai0643' 'category generator refuel fixed diesel special access n a vendor acas problem description dxl02686 rbs generator fuel low fieldreplaceableunit=sau alarmport=13 rbs generator fuel low tx hawkins cell site us 80 hawkins 2024 08 08 01 55 32 2024 08 08 01 58 24 04 32 33'

Evaluation

Metrics

Label	Accuracy
all	0.625

Uses

Direct Use for Inference

First install the SetFit library:

pip install setfit

Then you can load this model and run inference.

from setfit import SetFitModel

# Download from the 🤗 Hub
model = SetFitModel.from_pretrained("edwsiew/phantom-dispatch-02")
# Run inference
preds = model("category generator refuel fixed diesel special access no vendor acas problem description rbs generator fuel low")

Training Details

Training Set Metrics

Training set	Min	Median	Max
Word count	16	168.2540	915

Label	Training Sample Count
0	14
1	49

Training Hyperparameters

• batch_size: (8, 8)
• num_epochs: (1, 1)
• max_steps: -1
• sampling_strategy: oversampling
• num_iterations: 20
• body_learning_rate: (2e-05, 2e-05)
• head_learning_rate: 2e-05
• loss: CosineSimilarityLoss
• distance_metric: cosine_distance
• margin: 0.25
• end_to_end: False
• use_amp: False
• warmup_proportion: 0.1
• seed: 42
• eval_max_steps: -1
• load_best_model_at_end: False

Training Results

Epoch	Step	Training Loss	Validation Loss
0.0032	1	0.31	-
0.1587	50	0.0308	-
0.3175	100	0.0131	-
0.4762	150	0.0023	-
0.6349	200	0.0056	-
0.7937	250	0.0009	-
0.9524	300	0.0003	-

Framework Versions

• Python: 3.12.0
• SetFit: 1.0.3
• Sentence Transformers: 3.0.1
• Transformers: 4.39.0
• PyTorch: 2.4.0+cu121
• Datasets: 2.21.0
• Tokenizers: 0.15.2

Citation

BibTeX

@article{https://doi.org/10.48550/arxiv.2209.11055,
    doi = {10.48550/ARXIV.2209.11055},
    url = {https://arxiv.org/abs/2209.11055},
    author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
    keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
    title = {Efficient Few-Shot Learning Without Prompts},
    publisher = {arXiv},
    year = {2022},
    copyright = {Creative Commons Attribution 4.0 International}
}