Latent Upscale

Enhance the latent upscale options in the img2img process in Automatic1111 to provide more flexibility and better image quality.

Motivation

The current img2img process lacks the ability to select different options for Latent Upscale.
The default latent upscale method ("bilinear") often produces blurry images.

This plugin introduces alternative interpolation methods for upscaling and offers different schedulers for the diffusion process, resulting in superior upscaled images. Moreover, this plugin expands the upscale options available in the Latent Space, surpassing those offered by the "Hires Fix" for the txt2img process.

Example Comparison

Original Image:

The default latent upscale (choose Resize mode "latent upscale"):

Latent Upscale Plugin (Upscale method: "nearest-exact", Scheduler: "simple")

Installation

You can find the "Latent Upscale" plugin in the Available section under the Extensions tab in the WebUI. Simply search for "Latent Upscale" in the extension search bar to locate it and click on "install".

Alternatively, you can install this plugin by navigating to Install from URL under the Extensions tab, and then pasting this repo's URL and clicking on Install.

Don't forget to go back to Installed and click on Apply.

Usage

At the bottom of the img2img tab, you can select the newly installed Latent Upscale script in the Script dropdown menu.

To benefit from these enhancements, make sure you have the "Just resize (latent upscale)" option selected for Resize mode. Additionally, all the parameters present in the user interface remain applicable, alongside the new options provided by this plugin in "Upscale Method" and "Scheduler".

Tips

Opting for "nearest", "nearest-exact", or "area" in the Upscale Method and "simple" or "normal" in the Scheduler can often lead to crisper images. However, don't hesitate to experiment with different choices to discover the best outcome.
If you want more details in the image, you can try to increase the number of steps in the diffusion process. However, this will also increase the time required to generate the image.

Technical Details

How Latent Upscale Works

The original image is first encoded into the latent space, which is upscaled by the correct factor before being fed into the diffusion (de-noising) process, and then decoded to the upscaled image.

Automatic1111's Default Implementation

There are two problems with the default implementation of the latent upscale process in Automatic111:

It uses the “bilinear” method, which often results in blurry upscaled images. It does not provide users with other options to choose from.
The number of steps users set was applied to the full scheduler, effectively reducing the steps for “de-noising”. For example, if the user sets the steps to be "30" and the de-noise strength is set to 0.4, the diffusion process will only run for 30*0.4 = 12 steps.

Improved Implementation

This plugin overrides the default init method for StableDiffusionProcessingImg2Img to include additional features:
1. It adds the option to choose the "Upscale Method" interpolation method when creating the latent image.
2. I turned on a fix that’s already in the code base:
```
opts.img2img_fix_steps = True
```
  This ensures that the diffusion process runs for the correct number of steps, as specified by the user. It is unclear why this was not the default setting for img2img.
This plugin assigns the sampler_noise_scheduler_override method for StableDiffusionProcessingImg2Img so that our custom schedulers can be used for the diffusion process.