xu-song's picture
add compression leaderboard
1b7fc74
raw
history blame
114 Bytes
from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("EleutherAI/gpt-neox-20b")