using with transformers
How to Use This Model With Transformers Library?
While Trying I Got The Following Error!
OSError: TheBloke/stable-vicuna-13B-GPTQ does not appear to have a file named pytorch_model.bin, tf_model.h5,
model.ckpt or flax_model.msgpack.
You can't use this model with transformers directly. For that you need the - HF model, but it will need 30GB VRAM or 15gb in 8bit.
You can use this model with text generation UI as described in the readme. Or you can use it with AutoGPTQ which is an extension for HF format. AutoGPTQ is still in development and not yet fully stable but people are using it successfully with models like this one.
You can't use this model with transformers directly. For that you need the - HF model, but it will need 30GB VRAM or 15gb in 8bit.
You can use this model with text generation UI as described in the readme. Or you can use it with AutoGPTQ which is an extension for HF format. AutoGPTQ is still in development and not yet fully stable but people are using it successfully with models like this one.
I'll Try it For Sure and Thanks For Your Lightning Reply btw...