use vllm==0.6.3 load this model，it generate the fowllowing error
#1
by
wc-llm
- opened
when I use vllm==0.6.3 load this model，it generate the fowllowing error
File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/parameter.py", line 133, in load_qkv_weight
assert param_data.shape == loaded_weight.shape
AssertionError
wc-llm
changed discussion status to
closed