Spaces:

xu-song
/

tokenizer-arena

Running

App Files Files Community

tokenizer-arena / requirements.txt

xu-song's picture

update

42127b1 4 months ago

No virus

96 Bytes

	transformers
	sentencepiece
	tiktoken
	icetk
	torch
	zhon
	nltk
	boto3==1.34.54
	ai2-olmo
	ipadic
	fugashi