Spaces:

xu-song
/

tokenizer-arena

Running

tokenizer-arena / vocab /amber /__init__.py

add amber and crystal_coder

5db13e0 8 months ago

120 Bytes

	from transformers import LlamaTokenizer

	tokenizer = LlamaTokenizer.from_pretrained("LLM360/Amber", revision="ckpt_356")