Steelskull

/

L3-Arcania-4x8b

like 22

Hi Steelskull! Decided to leave feedback, since the model has very intriguing replies. I used a quantized GGUF 5_K_M version. Mostly tested with Koboldcpp v1.64 but also v1.64.1 and lmstudio v0.2.22. It should also be said that I used different sys prompts, from my own to something more standard.

1. Side notes:

• Long-form writing indeed. During RP somewhere around 250-400+ tokens per reply, but the length is justified by the level of details. For those who love to read, the narrative it provides is really enjoyable, or use TTS.
• Narrative focused: The level of narrative quality is impressive. It is able to immerse the user in the scene itself, in the moment, not only the character's actions in some bubble space. Probably one of the best among all smaller models I've tried so far. This model doesn't shy away from vividly describing and interacting with surroundings, which slow down the pacing and creates a sense of real time interaction. Even with that amount of text, it doesn't hallucinate logic, although some small details may seem strange, maybe due lower quants or untuned ST settings, but the fact that the model provides this small details is impressive.
• It works better when it has enough context from the user input to work with and not 15 t of input to generate 300-400 t of output. It can be shortened slightly by adding to sys prompt something like "Prioritize exclusively responses that are VERY SHORT in length." or something similar, to make it around 180-260 tokens (with the characters that require shorter messages).
• The model feels really smart enough and behaves realistically, without exaggerated emotions... which some may like and some may not.
• It may require markdown editing multiple times when mixing both asterisks with quotes. But separately (only * or "), it mostly works fine. Works best with quotes.
• The model has a wide range of behaviors and able to be violent depending on the character. So it doesn't stay only on positive traits, like some other models do, although it also isn't drawn to unwarranted cruelty.
• Sensitive to symbols inside system prompt; other users may not encounter this. It doesn't like when sys prompt uses [ ] and may finish reply with something like '....] and then fake link'. Or use of ``` with some OOC thoughts; with literally no reason. For now this issues encountered only during the first message tho.
• Seems confident with simple math or logic puzzles.
• Since I tested everything else I decided to test erp too. I would say it was very descriptive and the model preferred more literature language than vulgar.
• Technical issue: Almost with each message and any kind of character, it generates EOS </s> at the end of reply. Yeah, 'almost', because sometimes it doesn't. After that, it stops generation, and as we know ST hides EOS, so not a big deal, but still.

2. I have attached a screenshot with examples that showcases its behavior with different characters.
   Agoness (meant to be playful/sadistic) - superb
   Seraphina (tender/supportive) - superb
   Shiroha (Blind/insecure) - Hmm, quite good, I believe I envisioned her as more stubborn, with some violent outbursts stemming from insecurities.
   Nexa (pragmatic and mocking motivator) - superb, good reasoning and not only humiliation or motivational quotes.
   I also tried Cindy Aurum from FFXV that intended to have like 70-150 tokens per reply, but the model doesn't manage to keep short messages with any prompt. As a simple 40t assistant in lmstudio_v0.2.22 when I asked a math questions it structured replies in a weird way making a lot of new lines, and didn't worked well, maybe it was trying to stretch its replies, dunno; and again backticks during the first message.

3. Strengths: reasoning, descriptive, knowledgeable, focused, creative, good storyteller. The vivid description of the surroundings and not only the character's actions, is what sets this model apart from others I tried so far. Slow paced approach. Knowing that this model works best with long replies, I would consider this as a strong side.
   Weaknesses: Well, I will not critique any aspect of its behavior since everyone has their own taste and I like its adequacy and believable behavior. EOS at the end of the message is the main technical issue. Tested with a standard L3 prompt.

In conclusion, this model has exceptional story writing capabilities and is interesting to talk with. I've added it to my personal list of the top models. Thank you for making this model, wish you luck with your next models and MoE merges!