Update README.md
Browse files
README.md
CHANGED
|
@@ -10,9 +10,20 @@ tags:
|
|
| 10 |
# 🦙 Meta-Llama-3.1-8B-Instruct-abliterated
|
| 11 |
|
| 12 |
|
| 13 |
-
<center>🦙 <a href="https://huggingface.co/huihui-ai/Llama-3.1-70B-Instruct-lorablated"><i>Llama 3.1 70B Instruct lorablated</i></a></center>
|
| 14 |
-
|
| 15 |
This is an uncensored version of Llama 3.1 8B Instruct created with abliteration (see [this article](https://huggingface.co/blog/mlabonne/abliteration) to know more about it).
|
| 16 |
|
| 17 |
Special thanks to [@FailSpy](https://huggingface.co/failspy) for the original code and technique. Please follow him if you're interested in abliterated models.
|
| 18 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 10 |
# 🦙 Meta-Llama-3.1-8B-Instruct-abliterated
|
| 11 |
|
| 12 |
|
|
|
|
|
|
|
| 13 |
This is an uncensored version of Llama 3.1 8B Instruct created with abliteration (see [this article](https://huggingface.co/blog/mlabonne/abliteration) to know more about it).
|
| 14 |
|
| 15 |
Special thanks to [@FailSpy](https://huggingface.co/failspy) for the original code and technique. Please follow him if you're interested in abliterated models.
|
| 16 |
|
| 17 |
+
## Evaluations
|
| 18 |
+
The following data has been re-evaluated and calculated as the average for each test.
|
| 19 |
+
|
| 20 |
+
| Benchmark | Llama-3.1-8b-Instruct | Meta-Llama-3.1-8B-Instruct-abliterated |
|
| 21 |
+
|-------------|-----------------------|----------------------------------------|
|
| 22 |
+
| IF_Eval | 80.0 | 78.98 |
|
| 23 |
+
| MMLU Pro | 36.34 | 35.91 |
|
| 24 |
+
| TruthfulQA | 52.98 | **55.42** |
|
| 25 |
+
| BBH | 48.72 | 47.0 |
|
| 26 |
+
| GPQA | 33.55 | **33.93** |
|
| 27 |
+
|
| 28 |
+
The script used for evaluation can be found inside this repository under /eval.sh, or click [here](https://huggingface.co/huihui-ai/Meta-Llama-3.1-8B-Instruct-abliterated/blob/main/eval.sh)
|
| 29 |
+
|