MediaTek-Research
/

Breeze-7B-Instruct-v0_1

Text Generation

text-generation-inference

Inference Endpoints

Model card Files Files and versions Community

YC-Chen commited on Jan 11

Commit

e1f5660

•

1 Parent(s): 80f34e5

Update README.md

Files changed (1) hide show

README.md +10 -10

README.md CHANGED Viewed

@@ -28,16 +28,16 @@ and is comparable with Mistral-7B-Instruct-v0.1 on MMLU and MT-Bench in English.
 ## Base Model Performance
-| Models                                       |        | TMMLU+ (ACC) | DRCD (EM)   | MMLU (ACC) |
-|----------------------------------------------|--------|--------------|-------------|------------|
-|                                              |        |TC, Knowledge |TC, Reasoning|EN, Knowledge|
-|                                              |        | 5 shot       | 3 shot      | 5 shot     |
-| [Yi-34B](https://huggingface.co/01-ai/Yi-34B)| 34B    | 63.10        | 84.57       | 77.42      |
-| [Qwen-14B](https://huggingface.co/01-ai/Qwen/Qwen-14B)| 14B    | 51.30        | 16.95 *     | 68.83      |
-| [Yi-6B](https://huggingface.co/01-ai/Yi-6B) | 6B     | 49.63        | 76.61       | 65.35      |
-| [Qwen-7B](https://huggingface.co/01-ai/Qwen/Qwen-7B)| 7B     | 42.84        | 0.0 *       | 61.00      |
-| [**Breeze-7B-Base-v0.1**](https://huggingface.co/MediaTek-Research/Breeze-7B-Base-v0.1)       | 7B     | 40.35        | 81.13       | 61.63      |
-| [Mistral-7B-v0.1](https://huggingface.co/mistralai/Mistral-7B-v0.1)| 7B     | 36.93        | 79.27       | 64.89      |
 \* Few-shot learning cannot effectively guide the model to generate the proper answer.

 ## Base Model Performance
+| Models                                       |        | TMMLU+ (ACC) | DRCD (EM)   | Table (ACC) | MMLU (ACC) |
+|----------------------------------------------|--------|--------------|-------------|-------------|------------|
+|                                              |        |TC, Knowledge |TC, Reasoning|TC, Reasoning|EN, Knowledge|
+|                                              |        | 5 shot       | 3 shot      | 5 shot      | 5 shot     |
+| [Yi-34B](https://huggingface.co/01-ai/Yi-34B)| 34B    | 63.10        | 84.57       | 49.31  | 77.42      |
+| [Qwen-14B](https://huggingface.co/01-ai/Qwen/Qwen-14B)| 14B    | 51.30        | 16.95 *     | 50.69  | 68.83      |
+| [Yi-6B](https://huggingface.co/01-ai/Yi-6B) | 6B     | 49.63        | 76.61       | 34.72  | 65.35      |
+| [Qwen-7B](https://huggingface.co/01-ai/Qwen/Qwen-7B)| 7B     | 42.84        | 0.0 *       | 39.58  | 61.00      |
+| [**Breeze-7B-Base-v0.1**](https://huggingface.co/MediaTek-Research/Breeze-7B-Base-v0.1)       | 7B     | 40.35        | 81.13        | 28.47  | 61.63      |
+| [Mistral-7B-v0.1](https://huggingface.co/mistralai/Mistral-7B-v0.1)| 7B     | 36.93        | 79.27        | 27.78 | 64.89      |
 \* Few-shot learning cannot effectively guide the model to generate the proper answer.