<|im_end|> showing up at the end of response, even though I already added it in custom stopping strings
@MarinaraSpaghetti Hello! The "<|im_end|>" still shows up at the end of some outputs (looking like the attached picture). It appears once every 5/10 messages or so, and if I don't edit it out, it will show up more and more often, until it shows up everytime. Also attached what my custom stopping strings box looks like. Maybe I wrote it wrong? Thanks in advance!
I'm using your "Chat MLCustom" as instruct template and story string
Hey @wevvie , thanks for reaching out! Is it possible that you're using DRY sampler? If that's the case, then you'll need to add these symbols to its Sequence Breakers: "<", "|", ">", "im", "_", "end". Looks like the model is trying to output the full <|im_end|>, but the repetition penalty from the DRY is causing it to crash. Let me know if that helps!
Here's also my full string for Sequence Breakers if you want to yoink it.
["\n", ":", """, "'","*", "USER:", "ASSISTANT:", "2137:", "Dottore:", "Narrator:", "Tartaglia", "Pierro", "Pantalone", "<|im_start|>", "<|im_end|>", "<", "|", ">", "im", "end", "", "start", "system", "USER", "ASSISTANT", "2137", "Dottore", "Narrator", "im_end", "im_start", "user", "assistant", "im_sep", "sep", "<|im_sep|>", "<|im_start|>user", "<|im_start|>assistant", "<|end|>", "", "[INST]", "[/INST]", "[", "]", "INST", "Il", "2", "1", "3", "7"]
Oh yeah, I totally forgot DRY could probably be effing things up! I'll give it a try later today and report back. Thanks!
Update: No more issues! Adding the strings to DRY fixed it entirely. Thanks a lot!
By the way, do you have some tips on how to avoid the character from replying as me? I didn't mention {{user}} (or any action/dialogue on my behalf) on the character cards, nor on the char's Initial Message, but sometimes, the outputs write my dialogue regardless. Maybe it's something about the persona description or the position it's inserted?
I'm sneaking into the topic. Do you have anything besides "<|im_end|>" in Custom Stopping Strings?
After a 'wipe' I lost my setup and I don't know if it's even worth throwing anything too complex in there if I'm only testing Reremix for now anyway.
Thank you so much for the update, @wevvie ! Glad to read it works fine now! As for the advice, sometimes the model will do it regardless what you do, unfortunately (depends on what data it was trained on), but you can reinforce the instruction as I recently did, by editing the fourth point like this:
4. Use {{user}}'s Persona to track {{user}}'s attributes; but do not write {{user}}'s dialogues, and do not react to {{user}}'s inner thoughts or private emotions unless they are explicitly expressed through dialogue, actions, or observable behavior.
This issue is barely existent for me, since I write long replies, giving the model a lot to work with, haha. Here's also how I wrote my Persona, it's inserted below {{char}}'s Personality:
Not much is known about 2137. She's Dottore's test subject, caught by the Fatui and brought by Omega segment to be experimented on. A Descender from Earth with amnesia, holding potential to wield all seven elements. Great at cooking, decent at drawing. Snarky and bratty, but kind. Adorable nerd who's very smart, albeit goofy. Talented in reading people.
2137 looks to be 1.72 meters tall; she is chubby, pale, has big breasts, wide hips, shoulder-length blonde hair, blue eyes framed by a pair of black glasses, prominent nose and three unique moles on the left cheek.
As for Custom Stopping Strings, here are the ones I'm using, @KerDisren :
["</s>", "\n{{user}}:", "\n{{char}}:", "\nSummary:", "\nInput", "\nUSER:", "\n### Instruction:", "<|im_end|>", "\nASSISTANT:", "\nUSER:", "\n</s>", "<|eot_id|>", "<|end|>", "<|im_start|>", "\nASSIST:", "<｜end▁of▁sentence｜>"]
Hope this helps!