it'd be awesome if nf4 was provided
#2
by
MayensGuds
- opened
i found it to be really good, faster than any other quant, with decent speed, and comparable quality
I don't have any experience with BNB NF4 outside using it with transformers, so I can't really help with this. Maybe asking in the original repo or one of the NF4 repos would be better?