BF16是不是依赖CUDA 11.7，我的机器是12.2
#7
by
Weiguo
- opened
下面编译不过，现在只能用8位量化，4090应该是可以跑半精度的
model = AutoModelForCausalLM.from_pretrained("Qwen-7B-Chat", device_map="auto", trust_remote_code=True, use_bf16=True).eval()
有没有可能支持
model = AutoModelForCausalLM.from_pretrained("Qwen-7B-Chat", device_map="cuda:0", trust_remote_code=True).half().cuda()
16bit量化都不行
TypeError: QWenLMHeadModel.init() got an unexpected keyword argument 'use_bf16'
TypeError: QWenLMHeadModel.init() got an unexpected keyword argument 'use_fp16'
16bit量化都不行
TypeError: QWenLMHeadModel.init() got an unexpected keyword argument 'use_bf16'TypeError: QWenLMHeadModel.init() got an unexpected keyword argument 'use_fp16'
要装某个依赖CUDA 11.7的东西
Weiguo
changed discussion status to
closed