WoW-1hr / added_tokens.json
TrevorAshby's picture
add tokenizer
aed14bc
raw
history blame contribute delete
64 Bytes
{"<bos>": 50257, "<eos>": 50258, "<sep>": 50259, "<pad>": 50260}