madebyollin commited on
Commit
583db6c
1 Parent(s): 8a11549

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +17 -4
README.md CHANGED
@@ -9,7 +9,20 @@ inference: false
9
 
10
  SDXL-VAE-FP16-Fix is the [SDXL VAE](https://huggingface.co/stabilityai/sdxl-vae), but modified to run in fp16 precision without generating NaNs.
11
 
12
- | VAE | Decoding in `float32` precision | Decoding in `float16` precision |
13
- | --------------------- | ------------------------------- | ------------------------------- |
14
- | SDXL-VAE | ✅ ![](./images/orig-fp32.png) | ⚠️ ![](./images/orig-fp16.png) |
15
- | SDXL-VAE-FP16-Fix | ✅ ![](./images/fix-fp32.png) | ✅ ![](./images/fix-fp16.png) |
 
 
 
 
 
 
 
 
 
 
 
 
 
 
9
 
10
  SDXL-VAE-FP16-Fix is the [SDXL VAE](https://huggingface.co/stabilityai/sdxl-vae), but modified to run in fp16 precision without generating NaNs.
11
 
12
+ | VAE | Decoding in `float32` / `bfloat16` precision | Decoding in `float16` precision |
13
+ | --------------------- | -------------------------------------------- | ------------------------------- |
14
+ | SDXL-VAE | ✅ ![](./images/orig-fp32.png) | ⚠️ ![](./images/orig-fp16.png) |
15
+ | SDXL-VAE-FP16-Fix | ✅ ![](./images/fix-fp32.png) | ✅ ![](./images/fix-fp16.png) |
16
+
17
+
18
+ ## Details
19
+
20
+ SDXL-VAE generates NaNs in fp16 because the internal activation values are too big:
21
+ ![](./images/activation-magnitudes.jpg)
22
+
23
+ SDXL-VAE-FP16-Fix was created by finetuning the SDXL-VAE to:
24
+ 1. keep the final output the same, but
25
+ 2. make the internal activation values smaller, by
26
+ 3. scaling down weights and biases within the network
27
+
28
+ There are slight discrepancies between the output of SDXL-VAE-FP16-Fix and SDXL-VAE, but the decoded images should be close enough for most purposes.