add directory structure compatible with transformers.js eef0569 Jo Kristian Bergum commited on Jan 22
clean up tokenizer and add [unused0] and [unused1] as special tokens 85b42e5 Jo Kristian Bergum commited on Jan 22