llama2-13b-megacode2_min100 / added_tokens.json
Andreas Koepf
add info about added_tokens
b38d1b5
raw
history blame contribute delete
142 Bytes
{
"<CLS>": 32000,
"<EOD>": 32002,
"<MASK>": 32003,
"<PAD>": 32004,
"<SEP>": 32001,
"<|im_end|>": 32006,
"<|im_start|>": 32005
}