Update tokenizer
#9
by
alvarobartt
HF staff
- opened
Description
Updates both tokenizer.json
and special_tokens_map.json
as in https://huggingface.co/meta-llama/Meta-Llama-3.1-8B-Instruct/commit/07eb05b21d191a58c577b4a45982fe0c049d0693
alvarobartt
changed pull request title from
Update special_tokens_map.json
to Update tokenizer
alvarobartt
changed pull request status to
merged
Chat template also was changed
https://huggingface.co/meta-llama/Meta-Llama-3.1-8B-Instruct/commit/8c22764a7e3675c50d4c7c9a4edb474456022b16#d2h-846292