Pretraining script
#1
by
risedangel
- opened
Hello
May you kindly provide the script for pretraining the models? A google colab would be best.
Or can you point me to a direction so that i can learn how to pretrain retnets?
Thank you in advance
best i can give you is the fairseq repo here, train code is really buggy and i dont even know how it runs
Any chance you can work out what dictionary / vocab file was used? Or can you provide the command line args you used to launch the fairseq training script?
the model trained here is based on wikitext, highly based on the textgen commands, only with --tpu since gcp gave me some