rasyosef commited on
Commit
1f54588
1 Parent(s): 4f1a0ba

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +8 -0
README.md CHANGED
@@ -82,6 +82,14 @@ print(output[0]['generated_text'])
82
 
83
  Note: If you want to use flash attention, call _AutoModelForCausalLM.from_pretrained()_ with _attn_implementation="flash_attention_2"_
84
 
 
 
 
 
 
 
 
 
85
  ## Demo
86
 
87
  Here's a colab notebook with a chat interface, you can use this to interact with the chat model.
 
82
 
83
  Note: If you want to use flash attention, call _AutoModelForCausalLM.from_pretrained()_ with _attn_implementation="flash_attention_2"_
84
 
85
+
86
+ ## Benchmarks
87
+
88
+ These benchmarks were run using EleutherAI's [lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness)
89
+
90
+ - **IFEval (Instruction Following Evaluation)**: IFEval is a fairly interesting dataset that tests the capability of models to clearly follow explicit instructions, such as “include keyword x” or “use format y”. The models are tested on their ability to strictly follow formatting instructions rather than the actual contents generated, allowing strict and rigorous metrics to be used.
91
+ - Score: **45.83**
92
+
93
  ## Demo
94
 
95
  Here's a colab notebook with a chat interface, you can use this to interact with the chat model.