ko-councel-gpt / added_tokens.json
qowlsdud's picture
Upload tokenizer
44a02f4
raw
history blame contribute delete
95 Bytes
{
"<|endoftext|>": 50256,
"<||bos||>": 50257,
"<||pad||>": 50258,
"<||unk||>": 50259
}