That's awesome! But...

#1
by Anonimus12345678902 - opened

That's awesome! But man, I wish I could download Theia-21B-v1-Q4_K_S.gguf to run on my 12 gig 3060 video card. Theia-21B-v1-Q3_K_M_M.gguf is too dumb and Theia-21B-v1-Q4_K_M_M.gguf is too slow. I think many people who have such a video card will agree with me.

Maybe try imatrix quants ? i1-Q3_K_L maybe.

https://huggingface.co/mradermacher/Theia-21B-v1-i1-GGUF

Sign up or log in to comment