Tokenizer load error
#2
by
kwanwoo02
- opened
I tried running the example code, but the tokenizer could not be loaded.
model_id = 'Bllossom/llama-3.2-Korean-Bllossom-3B'
tokenizer = AutoTokenizer.from_pretrained(model_id) #error
Exception: data did not match any variant of untagged enum ModelWrapper at line 1251003 column 3
I expected to use a tokenizer like "Bllossom/llama-3.1-Korean-Bllossom-Vision-8B" , so I tried it and it worked.
But I think you need to check the tokenizer of "Bllossom/llama-3.2-Korean-Bllossom-3B".
I hope you try using the latest transformers version
pip install -U transformers