xlm-roberta-flash-implementation / tokenizer_config.json
michael-guenther's picture
add tokenizer class
3d87c79
raw
history blame
74 Bytes
{
"model_max_length": 512,
"tokenizer_class": "XLMRobertaTokenizer"
}