Model Size - Can't even run fp16 on 24gb VRAM
#3
by
jdc4429
- opened
Would it be possible to create a variant that fits in 24gb vram? Possibly keep some of the fp16 layers but not all to get it to fit along with a text encoder. I have seen this done with the Flux Dev model except kept some fp32 layers. Works really well and maximizes available VRAM. The fp8 model works really well for images and it's much faster but not so great on text so far in my tests up to 8 steps. I have added the fp8 variant to my website which is https://aiimagecentral.com under the Flux tab. Will see how it does versus all the other models.
Could you also explain the differences between 3.0 and 3.1? Is it worth getting 3.0 as well?