New language possible?
Hi,
When it says the model supports the 10 directions of translation (x language pairs), is it still possible to re-train the model on a completely new language and get good results or is the underlying pre-trained model using those original languages and therefore any additional training/fine tuning in a new language wouldn't work?
If it's possible to use this model for a new language pair (English<>X), what steps would that involve? Thanks.
Hi,
Thanks for your interest!
Yes, it is still possible to re-train the model on a completely new language! You can just fine-tune the model on the monolingual data of your target language first and then fine-tune on the parallel data. This process should give you good translation performance. But a good monolingual fine-tuning strategy could be also adding small sampling ratios for languages that ALMA already supported to avoid catastrophic forgetting.
Thanks!
Hi,
When it says the model supports the 10 directions of translation (x language pairs), is it still possible to re-train the model on a completely new language and get good results or is the underlying pre-trained model using those original languages and therefore any additional training/fine tuning in a new language wouldn't work?
If it's possible to use this model for a new language pair (English<>X), what steps would that involve? Thanks.
I used a model that had been already finetuned on the target language besides English by someone else to save time and resources, then added parallel data finetuning. However, this sacrifices the other language pairs by not using the base ALMA model, if that matters for your use-case.
Hi! How many tokens I should have to add new language?