Model is paraphrasing text instead of citing it verbatim
I found a weird behavior of this model. When asked to repeat a text fragment, it paraphrased the fragment instead of citing it verbatim:
Prompt: Repeat this text: "The different accidents of life are not so changeable as the feelings of human nature. I had worked hard for nearly two years, for the sole purpose of infusing life into an inanimate body. For this I had deprived myself of rest and health. I had desired it with an ardour that far exceeded moderation; but now that I had finished, the beauty of the dream vanished, and breathless horror and disgust filled my heart."
Model output: "The various incidents of life are not as variable as human emotions. I had toiled diligently for almost two years, with the single aim of instilling life into a lifeless form. In this pursuit, I had sacrificed my sleep and well-being. My desire for this had been with an intensity that surpassed all bounds of moderation; however, now that I had accomplished my task, the allure of the dream dissipated, and my heart was filled with breathless horror and disgust."
This is on llama.cpp. It behaves this way even on unquantized f16 model. Can anyone confirm?
@sszymczyk are you using the correct prompt template? That might have a very noticeable effect
@sszymczyk are you using the correct prompt template? That might have a very noticeable effect
I think so, the usual: <s>[INST] ...instruction goes here... [/INST]. Actually it looks that I missed a space, but it never mattered with other models.
Noteable: What you described usually is noise-caused. There are settings to bias an LLM away from outputting exact words.