File size: 162 Bytes
f331792
 
 
 
1b7fc74
 
 
1
2
3
4
5
6
7
"""
这里有个 emoji.json 是干嘛的?
"""

from transformers import AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("abeja/gpt-neox-japanese-2.7b")