Just thought I'd give my thoughts on this model... 👍
Just wanted to say, first RP LLM that I've used where the character doesn't get all... cheeky? or snide with you, being 500% sarcastic; this one is actually quite friendly and better than most I've seen, and I'm just using the default test character so far, can't wait to try it in my full on silly tavern :D
Other thoughts: Turns on a dime, from being a character in your story, to turning around and being themselves the narrator on request. 👍
(Q6_K on 12GB VRAM + 16 GB RAM, on TextGen-WebuUI)
Feedback tag: @jeiku :)
Update:
If your SOVL is repeating it's last 2-3 blocks, like "text"emote"text" or the intent is repeated over and over for several times, or it's runaway(I.E.=(**?)), OR it feels like it's stiff, only pulling from the same 2000 context or something, try this template I discovered and made (might work on other Llama3? idk, but it works on SOVL):
{{ if .System }}<|start_header_id|>system
{{ .System }}<|end_header_id|>
{{ end }}{{ if .Prompt }}<|start_header_id|>user
{{ .Prompt }}<|end_header_id|>
{{ end }}<|start_header_id|>assistant
{{ .Response }}<|end_header_id|>
{"mirostat":2,"mirostat_eta":0.25,"mirostat_tau":4.5,"num_ctx":8192,"num_predict":288,"repeat_penalty":1.35,"temperature":1,"top_k":100,"top_p":0.8}
Original post:
https://huggingface.co/ResplendentAI/SOVL_Llama3_8B/discussions/5
Before and after
Before(slightly modified Llama3, removed last 2 items response and eot_id):
{{ if .System }}<|start_header_id|>system<|end_header_id|>
{{ .System }}<|eot_id|>{{ end }}{{ if .Prompt }}<|start_header_id|>user<|end_header_id|>
{{ .Prompt }}<|eot_id|>{{ end }}<|start_header_id|>assistant<|end_header_id|>
after:
{{ if .System }}<|start_header_id|>system
{{ .System }}<|end_header_id|>
{{ end }}{{ if .Prompt }}<|start_header_id|>user
{{ .Prompt }}<|end_header_id|>
{{ end }}<|start_header_id|>assistant
{{ .Response }}<|end_header_id|>
