Hugging Face's logo

Spaces:
xu-song
/
tokenizer-arena
Running

App Files Community
1
tokenizer-arena

Commit History

update
83a2e6b

xu-song commited on

add more tokenizers
4cbb545

xu-song commited on

update
299dc31

xu-song commited on

fix typo
efd894c

xu-song commited on

update
b0c41e0

xu-song commited on

add doc
70c093d

xu-song commited on

add doc
47e1616

xu-song commited on

update
0d55475

xu-song commited on

update gradio to 4.38.1
8167cc2

xu-song commited on

add compression_rate details
a4208a2

xu-song commited on

remove git lfs
e411220

xu-song commited on

update
7c73423

xu-song commited on

fix lru_cache and fix qwen_vocab
9d1b27e

xu-song commited on

launch with queue
baf4d1e

xu-song commited on

add access token
f1b4ae2

xu-song commited on

remove lru_cache: has some bugs
f652c69

xu-song commited on

update
97354e0

xu-song commited on

update
d3c1316

xu-song commited on

update
5a684a6

xu-song commited on

update
42127b1

xu-song commited on

remove vocabs; update compression_app; add character_app;
2bd606a

xu-song commited on

update
f331792

xu-song commited on

update
9558ae0

xu-song commited on

add compression leaderboard
1b7fc74

xu-song commited on

update compress rate
367a536

xu-song commited on

update compress rate
988921c

xu-song commited on

update
7d2062e

xu-song commited on

add grok mixtral
480ae5d

xu-song commited on

add compress rate
814ee6b

xu-song commited on

add zephyr
a6aee1d

xu-song commited on

requi
6b70021

xu-song commited on

requi
510279b

xu-song commited on

requi
7b522e7

xu-song commited on

config python version
11379e2

xu-song commited on

add xlm-roberta
057bc67

xu-song commited on

add amber and crystal_coder
5db13e0

xu-song commited on

add character glm
f0f84b2

xu-song commited on

update
f02dd94

xu-song commited on

fix PyO3PanicException
2461705

xu-song commited on

fix unicode error: 'unicodeescape' codec can't decode bytes in position 602-608: unknown Unicode character name
bce41d0

xu-song commited on

fix fastchat_t5_3b
c766a08

xu-song commited on

fix tiktoken special tokens
adcfb97

xu-song commited on

add aya
44c3329

xu-song commited on

fix olmo
2442c83

xu-song commited on

add olmo tokenizer
bbefe94

xu-song commited on

update
24b4aa5

xu-song commited on

update
1f833af

xu-song commited on

fix tiktoken
a6c67ec

xu-song commited on

fix gemma_7b
7011963

xu-song commited on

add gemma_7b
9c8ace5

xu-song commited on