5fa1a76
1
2
3
token A part of a sentence, usually a word, but can also be a subword (non-common words are often split in subwords) or a punctuation symbol.
