abhinavnmagic
commited on
Commit
•
b7a4623
1
Parent(s):
4fbce56
Update README.md
Browse files
README.md
CHANGED
@@ -49,11 +49,12 @@ Model evaluation metrics and results.
|
|
49 |
|
50 |
| Benchmark | Metric | Llama-2-7b-evolcodealpaca | Llama-2-7b-pruned50-retrained-evolcodealpaca |
|
51 |
|------------------------------------------------|---------------|-------------|-------------------------------|
|
52 |
-
| [HumanEval](https://arxiv.org/abs/2107.03374) | pass@1 |
|
53 |
|
54 |
## Model Training Details
|
55 |
|
56 |
-
|
|
|
57 |
|
58 |
## Help
|
59 |
|
|
|
49 |
|
50 |
| Benchmark | Metric | Llama-2-7b-evolcodealpaca | Llama-2-7b-pruned50-retrained-evolcodealpaca |
|
51 |
|------------------------------------------------|---------------|-------------|-------------------------------|
|
52 |
+
| [HumanEval](https://arxiv.org/abs/2107.03374) | pass@1 | 32.03 | 32.7 |
|
53 |
|
54 |
## Model Training Details
|
55 |
|
56 |
+
This model was obtained by sparse-tranfer of the sparse foundational model [Llama-2-7b-pruned50-retrained](https://huggingface.co/neuralmagic/Llama-2-7b-pruned50-retrained) on 60% of the [evolcodealpaca](https://huggingface.co/datasets/theblackcat102/evol-codealpaca-v1) dataset.
|
57 |
+
Training was perfomerd for 2 epochs and used the [SquareHead](https://arxiv.org/abs/2310.06927) knowledge distillation with [Llama-2-7b-evolcodealpaca](https://huggingface.co/neuralmagic/Llama-2-7b-evolcodealpaca) as teacher.
|
58 |
|
59 |
## Help
|
60 |
|