bartowski

/

Qwen2.5-14B_Uncensored_Instruct-GGUF

like 30

Thanks.

I tried Q6_K_L. Unfortunately this too runs into bad incohrent runaway generation.

I tested with the non-censored version, which runs really well. But this does not.

So I downloaded this model from here, https://huggingface.co/SicariusSicariiStuff/Qwen2.5-14B_Uncensored_Instruct after the fix, and made a gguf and Q6 quantization myself to doublecheck, sir. My own quantized model gives me exactly the same problem.

I humbly posit, something is wrong then with probably the original model, or llama.cpp, or how I am using it.
I lack the knowledge to identify the issue as my knowledge on these topics is skin deep. I am sorry if I missed something important, please forgive me.

It may be that this model also requires this incoming fix that is for FIM token support:

https://github.com/ggerganov/llama.cpp/pull/9609

Maybe qwen2.5 just does weird stuff

Is there any plan to fix the problem? I tried several models and they all had EOG token problem. The response stream won't stop.

I believe the problem was with the original https://huggingface.co/SicariusSicariiStuff/Qwen2.5-14B_Uncensored_Instruct (link now defunct).

I quantized them myself with the same results.

And I checked, this is the only attempt at uncensoring the 14B model.
This explains why all of the quantized models have the same runaway problem.

I have told the publisher of the model, who is no doubt far more intelligent than I to have attempted to uncensor it, during the same time as my last post here. He has taken it offline since then (hence the link is now defunct). I do not know if his time will permit him to create another one or diagnose the problem.

I think we will need to wait until someone as knowledgeable makes another attempt to uncensor it. As I do not know anyone in this field I can't say if anyone else is trying that at the moment.

So this explains why I couldn't make it work.