Commit
•
46d8c5a
1
Parent(s):
4fbce56
Update README.md (#1)
Browse files- Update README.md (b7a46230379d11bb761559221e6aaaad4a0f7b55)
Co-authored-by: Abhinav Agarwalla <abhinavnmagic@users.noreply.huggingface.co>
README.md
CHANGED
@@ -49,11 +49,12 @@ Model evaluation metrics and results.
|
|
49 |
|
50 |
| Benchmark | Metric | Llama-2-7b-evolcodealpaca | Llama-2-7b-pruned50-retrained-evolcodealpaca |
|
51 |
|------------------------------------------------|---------------|-------------|-------------------------------|
|
52 |
-
| [HumanEval](https://arxiv.org/abs/2107.03374) | pass@1 |
|
53 |
|
54 |
## Model Training Details
|
55 |
|
56 |
-
|
|
|
57 |
|
58 |
## Help
|
59 |
|
|
|
49 |
|
50 |
| Benchmark | Metric | Llama-2-7b-evolcodealpaca | Llama-2-7b-pruned50-retrained-evolcodealpaca |
|
51 |
|------------------------------------------------|---------------|-------------|-------------------------------|
|
52 |
+
| [HumanEval](https://arxiv.org/abs/2107.03374) | pass@1 | 32.03 | 32.7 |
|
53 |
|
54 |
## Model Training Details
|
55 |
|
56 |
+
This model was obtained by sparse-tranfer of the sparse foundational model [Llama-2-7b-pruned50-retrained](https://huggingface.co/neuralmagic/Llama-2-7b-pruned50-retrained) on 60% of the [evolcodealpaca](https://huggingface.co/datasets/theblackcat102/evol-codealpaca-v1) dataset.
|
57 |
+
Training was perfomerd for 2 epochs and used the [SquareHead](https://arxiv.org/abs/2310.06927) knowledge distillation with [Llama-2-7b-evolcodealpaca](https://huggingface.co/neuralmagic/Llama-2-7b-evolcodealpaca) as teacher.
|
58 |
|
59 |
## Help
|
60 |
|