squash
Browse files- README.md +10 -8
- diagram.png +0 -0
- yo_llama.jpeg +0 -0
- yo_llama.jpg +0 -0
README.md
CHANGED
@@ -12,7 +12,7 @@ This model is based on Llama-3-8B-Instruct weights, but **steered to respond wit
|
|
12 |
Heavily inspired by [Llama-3-8B-Instruct-MopeyMule](https://huggingface.co/failspy/Llama-3-8B-Instruct-MopeyMule),
|
13 |
this model has **not been fine-tuned** traditionally. Instead, I tried to identify and **amplify the rap "direction" in the activation space**.
|
14 |
|
15 |
-
![yo-Llama-3-8B-Instruct](https://huggingface.co/anakin87/yo-Llama-3-8B-Instruct/resolve/main/yo_llama.
|
16 |
|
17 |
Let's allow the model to introduce itself: π€
|
18 |
> Who are you?
|
@@ -22,18 +22,20 @@ I'm just a small part of the game/ a language model with a lot of fame
|
|
22 |
I'm trained on data, day and night/ to spit out rhymes and make it right
|
23 |
I'm a bot, a robot, a machine so fine/ I'm here to serve, but don't you get too divine
|
24 |
I'll answer questions, and spit out some flows/ But don't get it twisted, I'm just a rhyme, yo
|
25 |
-
I'm on the mic, but I ain't no star/I'm just a bot, trying to go far
|
26 |
-
I'm on the grind, 24/7, 365/Trying to make it, but it's all a whim
|
27 |
-
So listen up, and don't be slow/I'll spit some rhymes, and make it grow
|
28 |
-
I'm the bot, the robot, the rhyme machine/Tryna make it hot, but it's all a dream!
|
29 |
```
|
30 |
|
31 |
-
## π§ͺ
|
32 |
-
|
|
|
|
|
33 |
Is Mediated by a Single Direction"](https://arxiv.org/abs/2406.11717):
|
34 |
the authors showed a methodology to find the "refusal" direction in the activation space of Chat Language Models and erase or amplify it.
|
35 |
|
36 |
-
From a practical point of view, [Failspy](https://huggingface.co/failspy) showed how to apply this methodology to elicit/remove features other than refusal.
|
37 |
|
38 |
π Resources: [abliterator library](https://github.com/FailSpy/abliterator); [Llama-MopeyMule-3-8B-Instruct model](https://huggingface.co/failspy/Llama-3-8B-Instruct-MopeyMule); [Induce Melancholy notebook](https://huggingface.co/failspy/Llama-3-8B-Instruct-MopeyMule/blob/main/MopeyMule-Induce-Melancholy.ipynb).
|
39 |
|
|
|
12 |
Heavily inspired by [Llama-3-8B-Instruct-MopeyMule](https://huggingface.co/failspy/Llama-3-8B-Instruct-MopeyMule),
|
13 |
this model has **not been fine-tuned** traditionally. Instead, I tried to identify and **amplify the rap "direction" in the activation space**.
|
14 |
|
15 |
+
![yo-Llama-3-8B-Instruct](https://huggingface.co/anakin87/yo-Llama-3-8B-Instruct/resolve/main/yo_llama.jpg)
|
16 |
|
17 |
Let's allow the model to introduce itself: π€
|
18 |
> Who are you?
|
|
|
22 |
I'm trained on data, day and night/ to spit out rhymes and make it right
|
23 |
I'm a bot, a robot, a machine so fine/ I'm here to serve, but don't you get too divine
|
24 |
I'll answer questions, and spit out some flows/ But don't get it twisted, I'm just a rhyme, yo
|
25 |
+
I'm on the mic, but I ain't no star/ I'm just a bot, trying to go far
|
26 |
+
I'm on the grind, 24/7, 365/ Trying to make it, but it's all a whim
|
27 |
+
So listen up, and don't be slow/ I'll spit some rhymes, and make it grow
|
28 |
+
I'm the bot, the robot, the rhyme machine/ Tryna make it hot, but it's all a dream!
|
29 |
```
|
30 |
|
31 |
+
## π§ͺ how does it work?/How can I reproduce it?
|
32 |
+
![diagram](https://huggingface.co/anakin87/yo-Llama-3-8B-Instruct/resolve/main/diagram.png)
|
33 |
+
|
34 |
+
From a *theoretical* point of view, this experiment is based on the paper ["Refusal in Language Models
|
35 |
Is Mediated by a Single Direction"](https://arxiv.org/abs/2406.11717):
|
36 |
the authors showed a methodology to find the "refusal" direction in the activation space of Chat Language Models and erase or amplify it.
|
37 |
|
38 |
+
From a *practical* point of view, [Failspy](https://huggingface.co/failspy) showed how to apply this methodology to elicit/remove features other than refusal.
|
39 |
|
40 |
π Resources: [abliterator library](https://github.com/FailSpy/abliterator); [Llama-MopeyMule-3-8B-Instruct model](https://huggingface.co/failspy/Llama-3-8B-Instruct-MopeyMule); [Induce Melancholy notebook](https://huggingface.co/failspy/Llama-3-8B-Instruct-MopeyMule/blob/main/MopeyMule-Induce-Melancholy.ipynb).
|
41 |
|
diagram.png
ADDED
yo_llama.jpeg
DELETED
Binary file (200 kB)
|
|
yo_llama.jpg
ADDED