robbinhan

robbinhan
Β·

AI & ML interests

None yet

Recent Activity

Organizations

None yet

robbinhan's activity

reacted to garrethlee's post with πŸ”₯ 20 days ago
view post
Post
1893
The latest o1 model from OpenAI is still unable to answer 9.11 > 9.9 correctly πŸ€”

A possible explanation? Tokenization - and our latest work investigates how it affects a model's ability to do math!

In this blog post, we discuss:
πŸ”’ The different ways numbers are tokenized in modern LLMs
πŸ§ͺ Our detailed approach in comparing these various methods
πŸ₯ͺ How we got a free boost in arithmetic performance by adding a few lines of code to the base Llama 3 tokenizer
πŸ‘‘ and a definitive, best tokenization method for math in LLMs!

Check out our work here: huggingface/number-tokenization-blog
  • 2 replies
Β·