Hugging Face's logo

ArliAI
/
Llama-3.1-8B-ArliAI-RPMax-v1.1

Text Generation
Transformers
Safetensors
llama
conversational
Eval Results
text-generation-inference
Inference Endpoints
Model card Files Community
2
OwenArli leaderboard-pr-bot commited on
Commit
dc7f5cd
•
1 Parent(s): 3e26f5f

Adding Evaluation Results (#1)

Browse files

- Adding Evaluation Results (9fa9c9d949ff4e20d3ec2708367827689c1ad4ce)


Co-authored-by: Open LLM Leaderboard PR Bot <leaderboard-pr-bot@users.noreply.huggingface.co>

Files changed (1) hide show
  1. README.md +109 -1
README.md CHANGED
@@ -1,5 +1,100 @@
1
  ---
2
  license: llama3
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
3
  ---
4
 
5
  # Llama-3.1-8B-ArliAI-RPMax-v1.1
@@ -63,4 +158,17 @@ You are [character]. You have a personality of [personality description]. [Descr
63
  {{ model_answer_1 }}<|eot_id|><|start_header_id|>user<|end_header_id|>
64
 
65
  {{ user_message_2 }}<|eot_id|><|start_header_id|>assistant<|end_header_id|>
66
- ```
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: llama3
3
+ model-index:
4
+ - name: Llama-3.1-8B-ArliAI-RPMax-v1.1
5
+ results:
6
+ - task:
7
+ type: text-generation
8
+ name: Text Generation
9
+ dataset:
10
+ name: IFEval (0-Shot)
11
+ type: HuggingFaceH4/ifeval
12
+ args:
13
+ num_few_shot: 0
14
+ metrics:
15
+ - type: inst_level_strict_acc and prompt_level_strict_acc
16
+ value: 63.59
17
+ name: strict accuracy
18
+ source:
19
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=ArliAI/Llama-3.1-8B-ArliAI-RPMax-v1.1
20
+ name: Open LLM Leaderboard
21
+ - task:
22
+ type: text-generation
23
+ name: Text Generation
24
+ dataset:
25
+ name: BBH (3-Shot)
26
+ type: BBH
27
+ args:
28
+ num_few_shot: 3
29
+ metrics:
30
+ - type: acc_norm
31
+ value: 28.79
32
+ name: normalized accuracy
33
+ source:
34
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=ArliAI/Llama-3.1-8B-ArliAI-RPMax-v1.1
35
+ name: Open LLM Leaderboard
36
+ - task:
37
+ type: text-generation
38
+ name: Text Generation
39
+ dataset:
40
+ name: MATH Lvl 5 (4-Shot)
41
+ type: hendrycks/competition_math
42
+ args:
43
+ num_few_shot: 4
44
+ metrics:
45
+ - type: exact_match
46
+ value: 11.33
47
+ name: exact match
48
+ source:
49
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=ArliAI/Llama-3.1-8B-ArliAI-RPMax-v1.1
50
+ name: Open LLM Leaderboard
51
+ - task:
52
+ type: text-generation
53
+ name: Text Generation
54
+ dataset:
55
+ name: GPQA (0-shot)
56
+ type: Idavidrein/gpqa
57
+ args:
58
+ num_few_shot: 0
59
+ metrics:
60
+ - type: acc_norm
61
+ value: 4.47
62
+ name: acc_norm
63
+ source:
64
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=ArliAI/Llama-3.1-8B-ArliAI-RPMax-v1.1
65
+ name: Open LLM Leaderboard
66
+ - task:
67
+ type: text-generation
68
+ name: Text Generation
69
+ dataset:
70
+ name: MuSR (0-shot)
71
+ type: TAUR-Lab/MuSR
72
+ args:
73
+ num_few_shot: 0
74
+ metrics:
75
+ - type: acc_norm
76
+ value: 5.31
77
+ name: acc_norm
78
+ source:
79
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=ArliAI/Llama-3.1-8B-ArliAI-RPMax-v1.1
80
+ name: Open LLM Leaderboard
81
+ - task:
82
+ type: text-generation
83
+ name: Text Generation
84
+ dataset:
85
+ name: MMLU-PRO (5-shot)
86
+ type: TIGER-Lab/MMLU-Pro
87
+ config: main
88
+ split: test
89
+ args:
90
+ num_few_shot: 5
91
+ metrics:
92
+ - type: acc
93
+ value: 28.35
94
+ name: accuracy
95
+ source:
96
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=ArliAI/Llama-3.1-8B-ArliAI-RPMax-v1.1
97
+ name: Open LLM Leaderboard
98
  ---
99
 
100
  # Llama-3.1-8B-ArliAI-RPMax-v1.1
 
158
  {{ model_answer_1 }}<|eot_id|><|start_header_id|>user<|end_header_id|>
159
 
160
  {{ user_message_2 }}<|eot_id|><|start_header_id|>assistant<|end_header_id|>
161
+ ```
162
+ # [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard)
163
+ Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_ArliAI__Llama-3.1-8B-ArliAI-RPMax-v1.1)
164
+
165
+ | Metric |Value|
166
+ |-------------------|----:|
167
+ |Avg. |23.64|
168
+ |IFEval (0-Shot) |63.59|
169
+ |BBH (3-Shot) |28.79|
170
+ |MATH Lvl 5 (4-Shot)|11.33|
171
+ |GPQA (0-shot) | 4.47|
172
+ |MuSR (0-shot) | 5.31|
173
+ |MMLU-PRO (5-shot) |28.35|
174
+