How to use the special reserved tokens, such as `<|reserved_special_token_0|>` for fine-tuning?
Besides a whole bunch of bug reports on GitHub and Reddit saying things like "the embeddings for these tokens are not trained", there does not seem to be any official documentation about using these.
The only real response I've seen is from this issue on GitHub where it's mentioned:
could one use some of these tokens in finetunes (instead of adding additional tokens and resizing the vocabulary)
Yes you can, this is why they were added -- to support more use-cases without requiring vocab resize.
What I'm wondering though is whether these tokens can be used for PEFT/LoRA use-cases without training the embeddings for these. Suppose I'd like my model to always output some special text in a particular format, so I can regex on its outputs. Say I expect my training samples to look like this:
<|reserved_special_token_10|>Special output from the model<|reserved_special_token_11|>
Can I use these special tokens? I'm asking because I'm doing QLoRA with this model, and I tried doing the above, and it appears that the model basically refuses to output any of these reserved tokens. Instead, I get the following output all the time:
�Special output from the model�
I also tried using tokenizer.add_special_tokens({"additional_special_tokens": ['reserved_special_token_10 'reserved_special_token_11']}) and (I guess expectedly) that doesn't seem to work either.
Any help is greatly appreciated.
would also like to know!