why do we need + "<|endoftext|>" when processing query and doccument
#3
by
Mengyao00
- opened
what if removing it? will it cause error or accuracy drop?
I think there will be accuracy drop here, as the model trained to use the EOS token as representation.
If you retrain/finetune it without adding the eos, (using avg pooling or simple last token) it should be fine.