MaziyarPanahi
commited on
Commit
•
aa6ac75
1
Parent(s):
edc75e6
Update README.md
Browse files
README.md
CHANGED
@@ -14,6 +14,32 @@ NeuralPipe-7B-slerp-v0.2 is a merge of the following models:
|
|
14 |
* [OpenPipe/mistral-ft-optimized-1227](https://huggingface.co/OpenPipe/mistral-ft-optimized-1227)
|
15 |
* [mlabonne/NeuralHermes-2.5-Mistral-7B](https://huggingface.co/mlabonne/NeuralHermes-2.5-Mistral-7B)
|
16 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
17 |
## 🧩 Configuration
|
18 |
|
19 |
```yaml
|
|
|
14 |
* [OpenPipe/mistral-ft-optimized-1227](https://huggingface.co/OpenPipe/mistral-ft-optimized-1227)
|
15 |
* [mlabonne/NeuralHermes-2.5-Mistral-7B](https://huggingface.co/mlabonne/NeuralHermes-2.5-Mistral-7B)
|
16 |
|
17 |
+
## Eval
|
18 |
+
```
|
19 |
+
| Groups |Version|Filter|n-shot| Metric | Value | |Stderr|
|
20 |
+
|------------------|-------|------|-----:|-----------|------:|---|-----:|
|
21 |
+
|ai2_arc |N/A |none | 0|acc | 0.7554|± |0.0406|
|
22 |
+
| | |none | 0|acc_norm | 0.7573|± |0.0332|
|
23 |
+
|mmlu |N/A |none | 0|acc | 0.6188|± |0.1472|
|
24 |
+
| - humanities |N/A |none | 0|acc | 0.5645|± |0.1686|
|
25 |
+
| - other |N/A |none | 0|acc | 0.6987|± |0.1098|
|
26 |
+
| - social_sciences|N/A |none | 0|acc | 0.7215|± |0.0887|
|
27 |
+
| - stem |N/A |none | 0|acc | 0.5208|± |0.1392|
|
28 |
+
|truthfulqa |N/A |none | 0|acc | 0.4746|± |0.0024|
|
29 |
+
| | |none | 0|bleu_max |26.7118|± |0.8092|
|
30 |
+
| | |none | 0|bleu_acc | 0.4957|± |0.0175|
|
31 |
+
| | |none | 0|bleu_diff | 3.1016|± |0.8065|
|
32 |
+
| | |none | 0|rouge1_max |53.1171|± |0.8499|
|
33 |
+
| | |none | 0|rouge1_acc | 0.5055|± |0.0175|
|
34 |
+
| | |none | 0|rouge1_diff| 4.0629|± |1.0345|
|
35 |
+
| | |none | 0|rouge2_max |39.1331|± |1.0068|
|
36 |
+
| | |none | 0|rouge2_acc | 0.4492|± |0.0174|
|
37 |
+
| | |none | 0|rouge2_diff| 3.7457|± |1.1652|
|
38 |
+
| | |none | 0|rougeL_max |49.8547|± |0.8818|
|
39 |
+
| | |none | 0|rougeL_acc | 0.5006|± |0.0175|
|
40 |
+
| | |none | 0|rougeL_diff| 3.6422|± |1.0540|
|
41 |
+
```
|
42 |
+
|
43 |
## 🧩 Configuration
|
44 |
|
45 |
```yaml
|