There are two types of tokenizers you can use with 🤗 Transformers: | |
[PreTrainedTokenizer]: a Python implementation of a tokenizer. |
There are two types of tokenizers you can use with 🤗 Transformers: | |
[PreTrainedTokenizer]: a Python implementation of a tokenizer. |