Update README.md
Browse files
README.md
CHANGED
@@ -23,6 +23,30 @@ The following models were included in the merge:
|
|
23 |
* [mlabonne/ChimeraLlama-3-8B-v3](https://huggingface.co/mlabonne/ChimeraLlama-3-8B-v3)
|
24 |
* [johnsnowlabs/JSL-MedLlama-3-8B-v2.0](https://huggingface.co/johnsnowlabs/JSL-MedLlama-3-8B-v2.0)
|
25 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
26 |
### Configuration
|
27 |
|
28 |
The following YAML configuration was used to produce this model:
|
|
|
23 |
* [mlabonne/ChimeraLlama-3-8B-v3](https://huggingface.co/mlabonne/ChimeraLlama-3-8B-v3)
|
24 |
* [johnsnowlabs/JSL-MedLlama-3-8B-v2.0](https://huggingface.co/johnsnowlabs/JSL-MedLlama-3-8B-v2.0)
|
25 |
|
26 |
+
### Evaluation
|
27 |
+
|
28 |
+
multimedqa (0 shot)</br>
|
29 |
+
| Tasks |Version|Filter|n-shot| Metric |Value | |Stderr|
|
30 |
+
|-------------------------------|-------|------|-----:|--------|-----:|---|-----:|
|
31 |
+
| - medmcqa |Yaml |none | 0|acc |0.6087|± |0.0075|
|
32 |
+
| | |none | 0|acc_norm|0.6087|± |0.0075|
|
33 |
+
| - medqa_4options |Yaml |none | 0|acc |0.6269|± |0.0136|
|
34 |
+
| | |none | 0|acc_norm|0.6269|± |0.0136|
|
35 |
+
| - anatomy (mmlu) | 0|none | 0|acc |0.6963|± |0.0397|
|
36 |
+
| - clinical_knowledge (mmlu) | 0|none | 0|acc |0.7585|± |0.0263|
|
37 |
+
| - college_biology (mmlu) | 0|none | 0|acc |0.7847|± |0.0344|
|
38 |
+
| - college_medicine (mmlu) | 0|none | 0|acc |0.6936|± |0.0351|
|
39 |
+
| - medical_genetics (mmlu) | 0|none | 0|acc |0.8200|± |0.0386|
|
40 |
+
| - professional_medicine (mmlu)| 0|none | 0|acc |0.7684|± |0.0256|
|
41 |
+
|stem |N/A |none | 0|acc_norm|0.6129|± |0.0066|
|
42 |
+
| | |none | 0|acc |0.6440|± |0.0057|
|
43 |
+
| - pubmedqa | 1|none | 0|acc |0.7480|± |0.0194|
|
44 |
+
|
45 |
+
|Groups|Version|Filter|n-shot| Metric |Value | |Stderr|
|
46 |
+
|------|-------|------|-----:|--------|-----:|---|-----:|
|
47 |
+
|stem |N/A |none | 0|acc_norm|0.6129|± |0.0066|
|
48 |
+
| | |none | 0|acc |0.6440|± |0.0057|
|
49 |
+
|
50 |
### Configuration
|
51 |
|
52 |
The following YAML configuration was used to produce this model:
|