Fine-tuning issues
Actually I'm learning fine tuning, so I scraped some data from financial website and tried to fine-tuning it, after fine tunning, I'm getting output something like this,
: How help me?\n : helped me to understand the market and the opportunities that were available. They helped me to understand the market and the opportunities that were available. They helped me to understand the market and the opportunities that were available. They helped me to understand the market and the opportunities that were available. They helped me to understand the market and the opportunities that were available. They helped me to understand the market and the opportunities that were available. They helped me to understand the market and the opportunities that were available. They helped me to understand the market and the opportunities that were available. They helped me to understand the market and the opportunities that were available. They helped me to understand the market and the opportunities that were available. They helped me to understand the market and the opportunities that were available. They helped me to understand the market and the opportunities that were available.
Have you seen it's repeatedly repeating, how to solve it, I have seen some notebook and tried to learn, can anyone help me please.
please help me ASAP, I'm been trying this from yesterday. Please help.
I have exactly the same thing. I used two different tutorials with different code both based on the fine-tuning in the falcon 7b blog post and one repeats additional questions and answers over and over (almost like it's returning one Q&A, then related Q&A). The other code I tried today repeats over and over like this too.
I'd really like some help and to see an answer on this too please and I'll update if I find a solution.
Frustratingly the first tutorial I followed showed a video with perfect inference afterwards and the code from checking looks identical.
If it's of interest there is a similar thread on chatgpt here: https://wordpress.org/support/topic/chatgpt-repeats-his-sentence-over-and-over-after-finde-tuning-a-model/ I will also be checking out embeddings.
This article also on chatgpt mentions could be to do with training data formating: https://community.openai.com/t/fine-tuned-a-davinci-model-repeats-phrases-in-answers/54737/4.