Without GPU...
Is there a GGUF version available somewhere?
Thanks for the suggestion, currently, I only noticed that AutoCoder_S (6.7B) is converted to GGUF version: https://huggingface.co/ukung/AutoCoder_S_6.7B-GGUF/tree/main.
I will try to convert the 33B model to GGUF as soon as possible. I will leave a message here after conversion, thanks!
Hey, please check here: https://huggingface.co/Bin12345/AutoCoder-Q4_K_M-GGUF.
This repo contains the method for generating GGUF version. And the Q4_K_M-GGUF version of AutoCoder. You can also adjust the quantization type on your own.
If you want to adjust the quantization type, please follow the instruction and you will see something like this:
Sign in with your own huggingface account and then adjust the parameters in the red box.
Hey, please check here: https://huggingface.co/Bin12345/AutoCoder-Q4_K_M-GGUF.
This repo contains the method for generating GGUF version. And the Q4_K_M-GGUF version of AutoCoder. You can also adjust the quantization type on your own.
Thank you very much.
Unfortunately when I try to create a Q8_0 quantization type I receive a long log after a few minutes of processing ending with the following error:
(OSError: Not enough free space to write 102760448 bytes\n\rWriting: 83%|\xe2\x96\x88\xe2\x96\x88\xe2\x96\x88\xe2\x96\x88\xe2\x96\x88\xe2\x96\x88\xe2\x96\x88\xe2\x96\x88\xe2\x96\x8e | 55.4G/66.7G [04:20<00:52, 213Mbyte/s]\n')
I don't have an idea where the process is missing the free space or what to do to solve the issue...
I think it might because there is no enough storage space. I suggest to check the storage first, thanks.
Thanks. Sorry for sounding so ignorant but the storage place of what? My computer..., or something else...?
The storage space of your computer. The model is large #_#.
Hmmm... Interesting. I succeeded to convert the model today without any space being freed on my computer. Perhaps it was some other problems and not my machine...
Thank you very much for your help anyway and for this useful link to the converter: https://huggingface.co/spaces/ggml-org/gguf-my-repo
Thanks！
