Ahmadzei's picture
added 3 more tables for large emb model
5fa1a76
If you have any chat models, you should set their tokenizer.chat_template attribute and test it using
[~PreTrainedTokenizer.apply_chat_template], then push the updated tokenizer to the Hub.