|
--- |
|
|
|
|
|
{} |
|
--- |
|
|
|
# Model Card for Phi 2 SlimOrca |
|
|
|
<!-- Provide a quick summary of what the model is/does. --> |
|
|
|
Phi 2 finetuned on SlimOrca-Dedup. This model was trained with the goal of giving Phi 2 the ablity to generate the EOS token together with being capable of doing beam search. It can also follow custom system prompts as shown in the example below. |
|
|
|
## Model Details |
|
|
|
## How to Get Started with the Model |
|
|
|
```python |
|
import torch |
|
import transformers |
|
|
|
model = transformers.AutoModelForCausalLM.from_pretrained( |
|
"miguelcarv/phi-2-slimorca", |
|
trust_remote_code=True, |
|
).to('cuda') |
|
tokenizer = transformers.AutoTokenizer.from_pretrained("microsoft/phi-2") |
|
|
|
|
|
SYSTEM_PROMPT = "You are an AI assistant. You will be given a task. You must generate a short and concise answer." |
|
input_text = f"""{SYSTEM_PROMPT} |
|
|
|
Instruction: Give me the first 5 prime numbers and explain what prime numbers are. |
|
Output:""" |
|
|
|
with torch.no_grad(): |
|
outputs = model.generate( |
|
tokenizer(input_text, return_tensors="pt")['input_ids'].to('cuda'), |
|
max_length=1024, |
|
num_beams = 3, |
|
eos_token_id = tokenizer.eos_token_id |
|
) |
|
print(tokenizer.decode(outputs[0], skip_special_tokens=True)) |
|
``` |
|
|
|
## Training Details |
|
|
|
- Trained for one epoch on SlimOrca-Dedup |
|
- Learning rate: 1e-5 |
|
- Cosine learning rate decay to 0 |
|
- Optimizer: AdamW |
|
- Batch size: 256 |
|
- Trained with mixed-precision bfloat16 |
|
|
