---
# For reference on model card metadata, see the spec: https://github.com/huggingface/hub-docs/blob/main/modelcard.md?plain=1
# Doc / guide: https://huggingface.co/docs/hub/model-cards
{}
---

# Model Card for Phi 2 SlimOrca

<!-- Provide a quick summary of what the model is/does. -->

Phi 2 finetuned on SlimOrca-Dedup. This model was trained with the goal of giving Phi 2 the ablity to generate the EOS token together with being capable of doing beam search. It can also follow custom system prompts as shown in the example below.

## Model Details

## How to Get Started with the Model

```python
import torch
import transformers

model = transformers.AutoModelForCausalLM.from_pretrained(
    "miguelcarv/phi-2-slimorca",
    trust_remote_code=True,
).to('cuda')
tokenizer = transformers.AutoTokenizer.from_pretrained("microsoft/phi-2")


SYSTEM_PROMPT = "You are an AI assistant. You will be given a task. You must generate a short and concise answer."
input_text = f"""{SYSTEM_PROMPT}

Instruction: Give me the first 5 prime numbers and explain what prime numbers are. 
Output:"""

with torch.no_grad():
    outputs = model.generate(
        tokenizer(input_text, return_tensors="pt")['input_ids'].to('cuda'),
        max_length=1024,
        num_beams = 3,
        eos_token_id = tokenizer.eos_token_id
    )
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```

## Training Details

 - Trained for one epoch on SlimOrca-Dedup
 - Learning rate: 1e-5
 - Cosine learning rate decay to 0
 - Optimizer: AdamW
 - Batch size: 256
 - Trained with mixed-precision bfloat16