Update README.md

README.md

@@ -9,7 +9,18 @@ language:
 
 A series of SAEs trained on embeddings from [nomic-embed-text-v1.5](https://huggingface.co/nomic-ai/nomic-embed-text-v1.5)
 
-The SAEs were trained using the [10BT sample of Fineweb-Edu](https://huggingface.co/datasets/enjalot/fineweb-edu-sample-10BT-chunked-500).
+The SAEs were trained on the 100BT sample of Fineweb-EDU, see an example of the [10BT sample of Fineweb-Edu](https://huggingface.co/datasets/enjalot/fineweb-edu-sample-10BT-chunked-500).
 
 Run the models or train your own with [Latent SAE](https://github.com/enjalot/latent-sae)
 
+# Training
+
+The models were trained using Modal Labs infrastructure with the command:
+```bash
+modal run train_modal.py --batch-size 512 --grad-acc-steps 4 --k 64 --expansion-factor 32
+```
+
+Error and dead latents charts can be seen here:
+![image/png](https://cdn-uploads.huggingface.co/production/uploads/631bce12bf1351ed2bd6bffe/GKPdI97ogF5tF709oYbbY.png)
+
+The training code is heavily copied from https://github.com/EleutherAI/sae