ChenWeiLi commited on
Commit
34f19ed
1 Parent(s): 2d0d901

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +24 -0
README.md CHANGED
@@ -23,6 +23,30 @@ The following models were included in the merge:
23
  * [mlabonne/ChimeraLlama-3-8B-v3](https://huggingface.co/mlabonne/ChimeraLlama-3-8B-v3)
24
  * [johnsnowlabs/JSL-MedLlama-3-8B-v2.0](https://huggingface.co/johnsnowlabs/JSL-MedLlama-3-8B-v2.0)
25
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
26
  ### Configuration
27
 
28
  The following YAML configuration was used to produce this model:
 
23
  * [mlabonne/ChimeraLlama-3-8B-v3](https://huggingface.co/mlabonne/ChimeraLlama-3-8B-v3)
24
  * [johnsnowlabs/JSL-MedLlama-3-8B-v2.0](https://huggingface.co/johnsnowlabs/JSL-MedLlama-3-8B-v2.0)
25
 
26
+ ### Evaluation
27
+
28
+ multimedqa (0 shot)</br>
29
+ | Tasks |Version|Filter|n-shot| Metric |Value | |Stderr|
30
+ |-------------------------------|-------|------|-----:|--------|-----:|---|-----:|
31
+ | - medmcqa |Yaml |none | 0|acc |0.6087|± |0.0075|
32
+ | | |none | 0|acc_norm|0.6087|± |0.0075|
33
+ | - medqa_4options |Yaml |none | 0|acc |0.6269|± |0.0136|
34
+ | | |none | 0|acc_norm|0.6269|± |0.0136|
35
+ | - anatomy (mmlu) | 0|none | 0|acc |0.6963|± |0.0397|
36
+ | - clinical_knowledge (mmlu) | 0|none | 0|acc |0.7585|± |0.0263|
37
+ | - college_biology (mmlu) | 0|none | 0|acc |0.7847|± |0.0344|
38
+ | - college_medicine (mmlu) | 0|none | 0|acc |0.6936|± |0.0351|
39
+ | - medical_genetics (mmlu) | 0|none | 0|acc |0.8200|± |0.0386|
40
+ | - professional_medicine (mmlu)| 0|none | 0|acc |0.7684|± |0.0256|
41
+ |stem |N/A |none | 0|acc_norm|0.6129|± |0.0066|
42
+ | | |none | 0|acc |0.6440|± |0.0057|
43
+ | - pubmedqa | 1|none | 0|acc |0.7480|± |0.0194|
44
+
45
+ |Groups|Version|Filter|n-shot| Metric |Value | |Stderr|
46
+ |------|-------|------|-----:|--------|-----:|---|-----:|
47
+ |stem |N/A |none | 0|acc_norm|0.6129|± |0.0066|
48
+ | | |none | 0|acc |0.6440|± |0.0057|
49
+
50
  ### Configuration
51
 
52
  The following YAML configuration was used to produce this model: