File size: 1,528 Bytes
4390f4c c4a1d81 a35dad5 c4a1d81 0f60e61 f83c306 a35dad5 4390f4c a35dad5 4390f4c f83c306 c4a1d81 d67c2dd 4390f4c a35dad5 d67c2dd a42fc92 a35dad5 f83c306 a35dad5 e843c58 a35dad5 02806bf a35dad5 e843c58 a35dad5 e843c58 a35dad5 e843c58 a35dad5 e843c58 a35dad5 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 |
---
library_name: peft
tags:
- code
- instruct
- llama2
datasets:
- HuggingFaceH4/no_robots
base_model: meta-llama/Llama-2-7b-hf
license: apache-2.0
---
### Finetuning Overview:
**Model Used:** meta-llama/Llama-2-7b-hf
**Dataset:** HuggingFaceH4/no_robots
#### Dataset Insights:
[No Robots](https://huggingface.co/datasets/HuggingFaceH4/no_robots) is a high-quality dataset of 10,000 instructions and demonstrations created by skilled human annotators. This data can be used for supervised fine-tuning (SFT) to make language models follow instructions better.
#### Finetuning Details:
With the utilization of [MonsterAPI](https://monsterapi.ai)'s [LLM finetuner](https://docs.monsterapi.ai/fine-tune-a-large-language-model-llm), this finetuning:
- Was achieved with great cost-effectiveness.
- Completed in a total duration of 39mins 4secs for 1 epoch using an A6000 48GB GPU.
- Costed `$1.313` for the entire epoch.
#### Hyperparameters & Additional Details:
- **Epochs:** 1
- **Cost Per Epoch:** $1.313
- **Total Finetuning Cost:** $1.313
- **Model Path:** meta-llama/Llama-2-7b-hf
- **Learning Rate:** 0.0002
- **Data Split:** 100% train
- **Gradient Accumulation Steps:** 4
- **lora r:** 32
- **lora alpha:** 64
#### Prompt Structure
```
<|system|> <|endoftext|> <|user|> [USER PROMPT]<|endoftext|> <|assistant|> [ASSISTANT ANSWER] <|endoftext|>
```
#### Train loss :
license: apache-2.0 |