File size: 1,173 Bytes
08f2b30 c193453 08f2b30 c193453 08f2b30 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 |
---
language:
- en
library_name: transformers
pipeline_tag: text-generation
---
# GenZ 13B v2
The instruction finetuned model with 4K input length. The model is finetuned on top of pretrained LLaMa2
## Inference
```python
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained("budecosystem/genz-13b-v2", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("budecosystem/genz-13b-v2", torch_dtype=torch.bfloat16)
inputs = tokenizer("The world is", return_tensors="pt")
sample = model.generate(**inputs, max_length=128)
print(tokenizer.decode(sample[0]))
```
Use the following prompt template
```
A chat between a curious user and an artificial intelligence assistant. The assistant gives helpful, detailed, and polite answers to the user's questions.
USER: Hi, how are you? ASSISTANT:
```
## Finetuning
```bash
python finetune.py
--model_name meta-llama/Llama-2-13b
--data_path dataset.json
--output_dir output
--trust_remote_code
--prompt_column instruction
--response_column output
```
Check the GitHub for the code -> [GenZ](https://github.com/BudEcosystem/GenZ) |