File size: 1,430 Bytes
b32428f |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 |
## llama3-alpaca Model
### Description
The llama3-alpaca model is a language model trained on vast amounts of text data. It can be used for various natural language processing tasks, including text generation, completion, and more.
### Inference Code (Using unsloth)
```python
from unsloth import FastLanguageModel
import torch
FastLanguageModel.for_inference(model) # Enable native 2x faster inference
# Define your prompt
prompt = "Continue the Fibonacci sequence."
# Provide input for the model
inputs = tokenizer(
[prompt],
return_tensors="pt"
).to("cuda")
# Generate output
outputs = model.generate(
**inputs,
max_new_tokens=64,
use_cache=True
)
# Decode the generated output
generated_text = tokenizer.batch_decode(outputs)
print(generated_text)
```
### Inference Code (HF model)
```python
from transformers import AutoTokenizer, AutoModelForCausalLM
# Load tokenizer and model
tokenizer = AutoTokenizer.from_pretrained("mohamed1ai/llama3-alpaca")
model = AutoModelForCausalLM.from_pretrained("mohamed1ai/llama3-alpaca")
# Define your prompt
prompt = "Continue the Fibonacci sequence."
# Tokenize the prompt
input_ids = tokenizer.encode(prompt, return_tensors="pt")
# Generate output
output = model.generate(input_ids, max_length=100, num_return_sequences=1)
# Decode the generated output
generated_text = tokenizer.decode(output[0], skip_special_tokens=True)
print(generated_text)
``` |