Add SetFit model

1_Pooling/config.json

@@ -0,0 +1,7 @@
+{
+  "word_embedding_dimension": 768,
+  "pooling_mode_cls_token": false,
+  "pooling_mode_mean_tokens": true,
+  "pooling_mode_max_tokens": false,
+  "pooling_mode_mean_sqrt_len_tokens": false
+}

README.md

@@ -0,0 +1,221 @@
+---
+library_name: setfit
+tags:
+- setfit
+- sentence-transformers
+- text-classification
+- generated_from_setfit_trainer
+metrics:
+- accuracy
+widget:
+- text: There’s been an early morning skate that’s been going on for years. They have
+    a Facebook page. I think its “6:30 am morning drop-in skate at lakeview” might
+    be on a break right now, but would be a good way to find some other drop in skates
+    or a way into some of the beer leagues. Mountain biking is a great year around
+    activity, and a good way to meet people. Stop into any of the shops and ask about
+    group rides.
+- text: So despite the minor audience, i think they can be quite damaging. Especially
+    after the burning of the tents. It's only the beginning (not that I am expecting
+    some form of January 6 style thing), they have been brewing for a while and have
+    now found an outlet.
+- text: My worry is that pretty much my entire social life is spent at a bar of some
+    sort, especially in the winter. So this month might suck. But I'm going to go
+    to my golf course's bar after work today with my 6pk of Craft non-alcoholic beer
+    so we'll see how that goes! Best of luck to you as well!
+- text: '@GayChemist @TylerDinucci It''s Wisconsin. I assume that "drinking fountains"
+    mean beer there.'
+- text: It's great! Spent some time at the brewery and got it there. Craft beer and
+    hot sauces are probably my two biggest vices lol.
+pipeline_tag: text-classification
+inference: true
+base_model: sentence-transformers/paraphrase-mpnet-base-v2
+model-index:
+- name: SetFit with sentence-transformers/paraphrase-mpnet-base-v2
+  results:
+  - task:
+      type: text-classification
+      name: Text Classification
+    dataset:
+      name: Unknown
+      type: unknown
+      split: test
+    metrics:
+    - type: accuracy
+      value: 0.0
+      name: Accuracy
+---
+
+# SetFit with sentence-transformers/paraphrase-mpnet-base-v2
+
+This is a [SetFit](https://github.com/huggingface/setfit) model that can be used for Text Classification. This SetFit model uses [sentence-transformers/paraphrase-mpnet-base-v2](https://huggingface.co/sentence-transformers/paraphrase-mpnet-base-v2) as the Sentence Transformer embedding model. A [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance is used for classification.
+
+The model has been trained using an efficient few-shot learning technique that involves:
+
+1. Fine-tuning a [Sentence Transformer](https://www.sbert.net) with contrastive learning.
+2. Training a classification head with features from the fine-tuned Sentence Transformer.
+
+## Model Details
+
+### Model Description
+- **Model Type:** SetFit
+- **Sentence Transformer body:** [sentence-transformers/paraphrase-mpnet-base-v2](https://huggingface.co/sentence-transformers/paraphrase-mpnet-base-v2)
+- **Classification head:** a [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance
+- **Maximum Sequence Length:** 512 tokens
+- **Number of Classes:** 6 classes
+<!-- - **Training Dataset:** [Unknown](https://huggingface.co/datasets/unknown) -->
+<!-- - **Language:** Unknown -->
+<!-- - **License:** Unknown -->
+
+### Model Sources
+
+- **Repository:** [SetFit on GitHub](https://github.com/huggingface/setfit)
+- **Paper:** [Efficient Few-Shot Learning Without Prompts](https://arxiv.org/abs/2209.11055)
+- **Blogpost:** [SetFit: Efficient Few-Shot Learning Without Prompts](https://huggingface.co/blog/setfit)
+
+### Model Labels
+| Label | Examples                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               |
+|:------|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| 3     | <ul><li>"I can relate to a lot of what you've described here. I call myself a beer snob because I enjoy trying new craft beers but won't usually go for any thing else, and I know what I like when it comes to beer. I'm currently on a hiatus because while I was doing well at having less for a couple of weeks, last weekend I went for 8/day two weekend days in a row. My doctor told me recently that 3 is a healthy limit, and I more than doubled that on those days so ... When it becomes a problem for you; When you're too sick the next day to do anything, etc, that's what eventually made me decide I want to shift my focus from drinking beer to everything else in life. It was fun while it lasted, may not be over for me entirely I just don't like where I was headed. Hope this helps."</li><li>'I’ve been trying out NA beer. They’re significantly lower calories and, well, non-alcoholic. It scratches the itch for me.'</li><li>"Yeah, I weigh myself daily and definitely notice how much even a couple hundred extra calories can stall my weight loss. It's been helpful in motivating myself: while a couple of beers after work doesn't seem like much, it's not worth it when I hop on the scale the next few days. Being strict also helped me choose healthier options. If I know a slice of cake has the same calories as a high-protein wrap but is more likely to make me feel tired and hungry later, it naturally becomes a lot less appealing."</li></ul> |
+| 1     | <ul><li>"These things happen sometimes, and that's what rescue meds are for. Today is the day I try to get out of the house and meet up with friends at a bar. Part of why I'm not drinking right now is because I'd skip going out and just drink beers at home. I want to socialize more. I'm a bit hesitant to go today though."</li><li>"Beer and boardgames in Heidelberg. It's an Event you can find on Meetup. Just 20 to 40 mostly international people meeting to play boardgames. No drinking beer needed, I drink apple juice. There is also a boardgame group meeting at the kolosseum in Heidelberg Thursdays at 17 uhr Hope you like those options"</li><li>'I go to a lot of motorcycle meetups, they are often at bars or breweries. My usual m.o. is to have one regular beer, and then switch over to Athletic Brews, a Non alcoholic beer. I like the taste, and it’s nice to be able to chill and drink with the other guys. I just really don’t want to risk riding home after having more than one alcoholic beer.'</li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                    |
+| 4     | <ul><li>'Like the title says, what are your ways of keeping track of bourbons and other spirits you’ve tried over the years? When it comes to beer I’ve used the app untappd to rate and discover new craft beers in the past and I’ve really enjoyed using something like that.'</li><li>'Enjoy it. S-189 is a great yeast. I would not call it a Cold IPA. I would just call it a tasty noble-dry-hopped pilsner. Prost! Click to expand...'</li><li>"I've got some super high gravity homebrews that are over 20 years old - some over 25. They've been kept properly. Last one I cracked was a few years ago (over 20 years old - an imperial stout with elderberries from my sister's farm) and it was sublime. I wouldn't bother with commercial American light lagers that are over six months old (they are so underhopped anyway and those hop oils deteriorate), but whatever floats your goat."</li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   |
+| 2     | <ul><li>"Taking one beer or glass of wine or so helps me relax and calm my thoughts. And it's tasty."</li><li>"Every person I know who works night shift has a drink or two after their shift just to fall asleep lol it might be a common thing. If it's in moderation and it's not ruining your health or interfering with your life, I'd say there's nothing wrong with having a beer or two to relax after work. After all you worked for it."</li><li>'Yes rdr2 is a really good game if you just want to relax, i come back from work sometimes order 2 beers, a burger, and fries and just enjoy the good music, scenery, and story'</li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  |
+| 5     | <ul><li>'Having a beer at a wine farm, ah bo gintsa bona will never leave me ????'</li><li>"As a java lover, I determined to locate out which quickly meals chain in Canada made the fine coffee... and the reply certainly took me via surprise. To get an concept of the great espresso in Canada, I tried the brew from three of the most famous espresso chains in the country. I stored it easy by means of sampling a small black espresso from Starbucks, Tim Hortons and McDonald's to discover out which one reigns supreme."</li><li>"Despite it being only 92 with 35% humidity, a notably comfortable summer evening, my beer is gone and I'd better go cook up some supper . Y'all have fun!"</li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   |
+| 0     | <ul><li>'It is 6:00 am. I’m preparing to head to the Muni with my father to celebrate the lives of my lifelong Browns fan uncle and grandfather who both passed away last week. Beers will be had, good times will be celebrated, and the Browns will kick ass. It is written. IT IS GAMEDAY MY DUDES.'</li><li>'I live in the "boonies" and beer is $14+ a can at any sporting event or concert I go to. Its wild.'</li><li>'Love it everytime! Best wings ever in my opinion especially the Carolina sweet & tangy. Bar quesadillas are to die for , lots of beer options for beer lovers like myself. & ALWAYS has the games on that I want to watch whether it’s basketball , baseball or football . My go to bar on the island .'</li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       |
+
+## Evaluation
+
+### Metrics
+| Label   | Accuracy |
+|:--------|:---------|
+| **all** | 0.0      |
+
+## Uses
+
+### Direct Use for Inference
+
+First install the SetFit library:
+
+```bash
+pip install setfit
+```
+
+Then you can load this model and run inference.
+
+```python
+from setfit import SetFitModel
+
+# Download from the 🤗 Hub
+model = SetFitModel.from_pretrained("bhaskars113/beer-model")
+# Run inference
+preds = model("@GayChemist @TylerDinucci It's Wisconsin. I assume that \"drinking fountains\" mean beer there.")
+```
+
+<!--
+### Downstream Use
+
+*List how someone could finetune this model on their own dataset.*
+-->
+
+<!--
+### Out-of-Scope Use
+
+*List how the model may foreseeably be misused and address what users ought not to do with the model.*
+-->
+
+<!--
+## Bias, Risks and Limitations
+
+*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
+-->
+
+<!--
+### Recommendations
+
+*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
+-->
+
+## Training Details
+
+### Training Set Metrics
+| Training set | Min | Median  | Max |
+|:-------------|:----|:--------|:----|
+| Word count   | 3   | 60.6875 | 260 |
+
+| Label | Training Sample Count |
+|:------|:----------------------|
+| 0     | 16                    |
+| 1     | 16                    |
+| 2     | 16                    |
+| 3     | 16                    |
+| 4     | 16                    |
+| 5     | 16                    |
+
+### Training Hyperparameters
+- batch_size: (16, 16)
+- num_epochs: (1, 1)
+- max_steps: -1
+- sampling_strategy: oversampling
+- num_iterations: 20
+- body_learning_rate: (2e-05, 2e-05)
+- head_learning_rate: 2e-05
+- loss: CosineSimilarityLoss
+- distance_metric: cosine_distance
+- margin: 0.25
+- end_to_end: False
+- use_amp: False
+- warmup_proportion: 0.1
+- seed: 42
+- eval_max_steps: -1
+- load_best_model_at_end: False
+
+### Training Results
+| Epoch  | Step | Training Loss | Validation Loss |
+|:------:|:----:|:-------------:|:---------------:|
+| 0.0042 | 1    | 0.2324        | -               |
+| 0.2083 | 50   | 0.1497        | -               |
+| 0.4167 | 100  | 0.0141        | -               |
+| 0.625  | 150  | 0.002         | -               |
+| 0.8333 | 200  | 0.0013        | -               |
+
+### Framework Versions
+- Python: 3.10.12
+- SetFit: 1.0.3
+- Sentence Transformers: 2.2.2
+- Transformers: 4.35.2
+- PyTorch: 2.1.0+cu121
+- Datasets: 2.16.1
+- Tokenizers: 0.15.1
+
+## Citation
+
+### BibTeX
+```bibtex
+@article{https://doi.org/10.48550/arxiv.2209.11055,
+    doi = {10.48550/ARXIV.2209.11055},
+    url = {https://arxiv.org/abs/2209.11055},
+    author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
+    keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
+    title = {Efficient Few-Shot Learning Without Prompts},
+    publisher = {arXiv},
+    year = {2022},
+    copyright = {Creative Commons Attribution 4.0 International}
+}
+```
+
+<!--
+## Glossary
+
+*Clearly define terms in order to be accessible across audiences.*
+-->
+
+<!--
+## Model Card Authors
+
+*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
+-->
+
+<!--
+## Model Card Contact
+
+*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
+-->

config.json

@@ -0,0 +1,24 @@
+{
+  "_name_or_path": "/root/.cache/torch/sentence_transformers/sentence-transformers_paraphrase-mpnet-base-v2/",
+  "architectures": [
+    "MPNetModel"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "bos_token_id": 0,
+  "eos_token_id": 2,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 768,
+  "initializer_range": 0.02,
+  "intermediate_size": 3072,
+  "layer_norm_eps": 1e-05,
+  "max_position_embeddings": 514,
+  "model_type": "mpnet",
+  "num_attention_heads": 12,
+  "num_hidden_layers": 12,
+  "pad_token_id": 1,
+  "relative_attention_num_buckets": 32,
+  "torch_dtype": "float32",
+  "transformers_version": "4.35.2",
+  "vocab_size": 30527
+}

config_sentence_transformers.json

@@ -0,0 +1,7 @@
+{
+  "__version__": {
+    "sentence_transformers": "2.0.0",
+    "transformers": "4.7.0",
+    "pytorch": "1.9.0+cu102"
+  }
+}

config_setfit.json

@@ -0,0 +1,4 @@
+{
+  "labels": null,
+  "normalize_embeddings": false
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a2459d7739cb9579600b5e2af33dace2c1e8dc91f40e29b0119d5b1988fc941c
+size 437967672

model_head.pkl

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4586b6f3813a0205697cc74f5dfd1461e6536f2df353d83253531493f6c8ba78
+size 37783

modules.json

@@ -0,0 +1,14 @@
+[
+  {
+    "idx": 0,
+    "name": "0",
+    "path": "",
+    "type": "sentence_transformers.models.Transformer"
+  },
+  {
+    "idx": 1,
+    "name": "1",
+    "path": "1_Pooling",
+    "type": "sentence_transformers.models.Pooling"
+  }
+]

sentence_bert_config.json

@@ -0,0 +1,4 @@
+{
+  "max_seq_length": 512,
+  "do_lower_case": false
+}

special_tokens_map.json

@@ -0,0 +1,51 @@
+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "cls_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "mask_token": {
+    "content": "<mask>",
+    "lstrip": true,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<pad>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "sep_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "[UNK]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer_config.json

@@ -0,0 +1,59 @@
+{
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "<pad>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "</s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "104": {
+      "content": "[UNK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "30526": {
+      "content": "<mask>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "<s>",
+  "clean_up_tokenization_spaces": true,
+  "cls_token": "<s>",
+  "do_basic_tokenize": true,
+  "do_lower_case": true,
+  "eos_token": "</s>",
+  "mask_token": "<mask>",
+  "model_max_length": 512,
+  "never_split": null,
+  "pad_token": "<pad>",
+  "sep_token": "</s>",
+  "strip_accents": null,
+  "tokenize_chinese_chars": true,
+  "tokenizer_class": "MPNetTokenizer",
+  "unk_token": "[UNK]"
+}