DanielDaniel / tokenizer_config.json
BapBap's picture
Upload tokenizer
b6acc5e
raw
history blame contribute delete
No virus
198 Bytes
{
"name_or_path": "/Users/baptistepannier/Scratch/tok",
"special_tokens_map_file": "/Users/baptistepannier/Scratch/tok/special_tokens_map.json",
"tokenizer_class": "PreTrainedTokenizerFast"
}