How to make it (Llama-2-13B-chat-GPTQ) work with Fastchat
#30
by
Vishvendra
- opened
this model is not loading in Fastchat, is there any GPTQ which is build with GPTQ
Vishvendra
changed discussion title from
How to make it (Llama-2-13B-chat-GPTQ) wot with Fastchat
to How to make it (Llama-2-13B-chat-GPTQ) work with Fastchat
With this model, the one in main is built with an GPTQ-for-LLaMa branch. And the ones in the other branches are made with AutoGPTQ.
The one in main - made with a very old version of GPTQ-for-LLaMa - will not work with FastChat, but all the others should.
In my more recent repos, all GPTQs are made with AutoGPTQ and should be compatible with FastChat.
Thanks, for the quick response. let me check the AutoGPTQ one.
I tried with AutoGPTQ and FastChat with no luck. Do you have any documentation/PR/Readme paper which have the process defined.
