Update README.md
Browse files
README.md
CHANGED
|
@@ -11,7 +11,7 @@ A series of SAEs trained on embeddings from [nomic-embed-text-v1.5](https://hugg
|
|
| 11 |
|
| 12 |
The SAEs were trained on the 100BT sample of Fineweb-EDU, see an example of the [10BT sample of Fineweb-Edu](https://huggingface.co/datasets/enjalot/fineweb-edu-sample-10BT-chunked-500).
|
| 13 |
|
| 14 |
-
Run the models or train your own with [Latent SAE](https://github.com/enjalot/latent-sae)
|
| 15 |
|
| 16 |
# Training
|
| 17 |
|
|
@@ -23,4 +23,3 @@ modal run train_modal.py --batch-size 512 --grad-acc-steps 4 --k 64 --expansion-
|
|
| 23 |
Error and dead latents charts can be seen here:
|
| 24 |
|
| 25 |
|
| 26 |
-
The training code is heavily copied from https://github.com/EleutherAI/sae
|
|
|
|
| 11 |
|
| 12 |
The SAEs were trained on the 100BT sample of Fineweb-EDU, see an example of the [10BT sample of Fineweb-Edu](https://huggingface.co/datasets/enjalot/fineweb-edu-sample-10BT-chunked-500).
|
| 13 |
|
| 14 |
+
Run the models or train your own with [Latent SAE](https://github.com/enjalot/latent-sae) which is heavily borrowing from https://github.com/EleutherAI/sae
|
| 15 |
|
| 16 |
# Training
|
| 17 |
|
|
|
|
| 23 |
Error and dead latents charts can be seen here:
|
| 24 |
|
| 25 |
|
|
|