fp8 inference

#26
by Melody32768 - opened

May I ask whether the input is quantified during inference

Sign up or log in to comment