|
--- |
|
base_model: unsloth/mistral-7b-v0.3-bnb-4bit |
|
language: |
|
- en |
|
license: apache-2.0 |
|
tags: |
|
- text-generation-inference |
|
- transformers |
|
- unsloth |
|
- mistral |
|
- trl |
|
- sft |
|
--- |
|
|
|
# Uploaded model |
|
|
|
This model was created by fine-tuning `unsloth/mistral-7b-v0.3-bnb-4bit` on [Short jokes dataset](https://www.kaggle.com/datasets/abhinavmoudgil95/short-jokes). |
|
So the only purpose of this model is the generation of cringe jokes. </br> |
|
Just write the first few words and get your joke. |
|
|
|
# Usage |
|
|
|
[Goodle Colab example](https://colab.research.google.com/drive/13N1O-fq-vjr8FUrsUU6f24fPpyf0ZwOS#scrollTo=UBSG1UTV85Vq) |
|
|
|
``` |
|
pip install transformers |
|
pip install --no-deps "trl<0.9.0" peft accelerate bitsandbytes |
|
``` |
|
``` |
|
from transformers import AutoTokenizer,AutoModelForCausalLM |
|
|
|
model = AutoModelForCausalLM.from_pretrained("SantaBot/Jokestral_4bit",) |
|
tokenizer = AutoTokenizer.from_pretrained("SantaBot/Jokestral_4bit") |
|
|
|
inputs = tokenizer( |
|
[ |
|
"My doctor" # YOUR PROMPT HERE |
|
], return_tensors = "pt").to("cuda") |
|
|
|
outputs = model.generate(**inputs, max_new_tokens = 64, use_cache = True) |
|
tokenizer.batch_decode(outputs) |
|
``` |
|
|
|
**The output should be something like** : </br> |
|
`['<s> My doctor told me I have to stop m4sturb4t1ng. I asked him why and he said ""Because I\'m trying to examine you.""\n</s>']` |
|
|
