V100上无法直接推理

by MonolithFoundation - opened Sep 6

Sep 6

如果V100上安装了flashattn2，但是实际上用不了flashatten2，代码便无法推理。
设置了config atten_implementation 为sdpa，代码不认这个还是flashattn2.

Sep 9

This comment has been hidden

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment