How to add new token or special token?

#16
by hahaMiao - opened

I just use add_tokens function, however, during the process of reasoning, neither regular tokens nor special tokens can be generated.

Sign up or log in to comment