hollowstrawberry
commited on
Commit
•
7f30f58
1
Parent(s):
c7e2ff9
Update README.md
Browse files
README.md
CHANGED
@@ -55,9 +55,9 @@ This guide is up to date with the best practices as of March 2023. One week is l
|
|
55 |
|
56 |
|
57 |
|
58 |
-
# Google
|
59 |
|
60 |
-
The easiest way to use Stable Diffusion is through Google
|
61 |
|
62 |
If you instead want to run it on your own computer, [scroll down â–¼](#install).
|
63 |
|
@@ -77,7 +77,7 @@ If you instead want to run it on your own computer, [scroll down â–¼](#install).
|
|
77 |
|
78 |
# Local Installation (Windows + Nvidia) <a name="install"></a>[â–²](#index)
|
79 |
|
80 |
-
To run Stable Diffusion on your own computer you'll need at least 16 GB of RAM and 4 GB of VRAM (preferably 8). I will only cover the case where you are running Windows 10/11 and using an NVIDIA graphics card series 16XX, 20XX or 30XX (though 10XX also work). My apologies to AMD, Linux, and Mac users, but their cases are harder to cover. If you don't meet the hardware requirements, you can just proceed with the Google
|
81 |
|
82 |
1. Get the latest release from [this page](https://github.com/EmpireMediaScience/A1111-Web-UI-Installer/releases).
|
83 |
|
@@ -108,7 +108,7 @@ If you followed the instructions above, the top of your page should look similar
|
|
108 |
|
109 |
![Top](images/top.png)
|
110 |
|
111 |
-
Here you can select your checkpoint and VAE. We will go over what these are and how you can get some. The
|
112 |
|
113 |
1. **Models** <a name="model"></a>[â–²](#index)
|
114 |
|
@@ -119,7 +119,7 @@ Here you can select your checkpoint and VAE. We will go over what these are and
|
|
119 |
* For photorealism go with [Deliberate](https://civitai.com/models/4823/deliberate). It can do almost anything, but specially photographs. Very intricate results.
|
120 |
* The [Uber Realistic Porn Merge](https://civitai.com/models/2661/uber-realistic-porn-merge-urpm) is self-explanatory.
|
121 |
|
122 |
-
If you're using the
|
123 |
|
124 |
If you're running the program locally, the models normally go into the `stable-diffusion-webui/models/Stable-diffusion` folder.
|
125 |
|
@@ -131,7 +131,7 @@ Here you can select your checkpoint and VAE. We will go over what these are and
|
|
131 |
|
132 |
Most checkpoints don't come with a VAE built in. The VAE is a small separate model, which "converts your image into human format". Without it, you'll get faded colors and ugly eyes, among other things.
|
133 |
|
134 |
-
If you're using the
|
135 |
|
136 |
There are practically only 3 different VAEs in circulation:
|
137 |
* [anything vae](https://huggingface.co/WarriorMama777/OrangeMixs/resolve/main/VAEs/orangemix.vae.pt), also known as the orangemix vae. All anime models use this.
|
@@ -159,7 +159,7 @@ Here you can select your checkpoint and VAE. We will go over what these are and
|
|
159 |
* `EasyNegative, worst quality, low quality, normal quality, child, painting, drawing, sketch, cartoon, anime, render, 3d, blurry, deformed, disfigured, morbid, mutated, bad anatomy, bad art`
|
160 |
|
161 |
* **EasyNegative:** <a name="promptneg"></a>The negative prompts above use EasyNegative, which is an *embedding* or "magic word" that encodes many bad things to make your images better. Otherwise you'd have to use a huge negative prompt.
|
162 |
-
* If you're using the
|
163 |
|
164 |
A comparison with and without these negative prompts including EasyNegative can be seen [further down â–¼](#matrixneg).
|
165 |
|
@@ -219,7 +219,7 @@ Here you can select your checkpoint and VAE. We will go over what these are and
|
|
219 |
|
220 |
![Extensions](images/extensions.png)
|
221 |
|
222 |
-
Here are some useful extensions. If you're using the
|
223 |
* [Image Browser (fixed fork)](https://github.com/aka7774/sd_images_browser) - This will let you browse your past generated images very efficiently, as well as directly sending their prompts and parameters back to txt2img, img2img, etc.
|
224 |
* [TagComplete](https://github.com/DominikDoom/a1111-sd-webui-tagcomplete) - Absolutely essential for anime art. It will show you the matching booru tags as you type. Anime models work via booru tags, and prompts without them usually don't work, so knowing them is godmode. Not all tags will work well in all models though, specially if they're rare.
|
225 |
* [ControlNet](https://github.com/Mikubill/sd-webui-controlnet) - A huge extension deserving of [its own guide â–¼](#controlnet). It lets you analyze any image and use it as an referene for your own image. Practically speaking, it can create any pose or environment you want.
|
@@ -236,7 +236,7 @@ LoRA or *Low-Rank Adaptation* is a form of **Extra Network** and the latest tech
|
|
236 |
|
237 |
Loras can represent a character, an artstyle, poses, clothes, or even a human face (though I do not endorse this). Checkpoints are usually capable enough for general work, but when it comes to specific details with little existing examples, you'll need a Lora. They can be downloaded from [civitai](https://civitai.com) or [elsewhere (NSFW)](https://gitgud.io/gayshit/makesomefuckingporn#lora-list) and are 144 MB by default, but they can go as low as 1 MB. Bigger Loras are not always better. They come in `.safetensors` format, same as most checkpoints.
|
238 |
|
239 |
-
Place your Lora files in the `stable-diffusion-webui/models/Lora` folder, or if you're using the
|
240 |
|
241 |
![Extra Networks](images/extranetworks.png)
|
242 |
|
@@ -250,7 +250,7 @@ As mentioned in [Generation Parameters â–²](#gen), normally you shouldn't go abo
|
|
250 |
|
251 |
You can download additional upscalers and put them in your `stable-diffusion-webui/models/ESRGAN` folder. They will then be available in Hires fix, Ultimate Upscaler, and Extras.
|
252 |
|
253 |
-
The
|
254 |
|
255 |
* A few notable ones can be [found here](https://huggingface.co/hollowstrawberry/upscalers-backup/tree/main/ESRGAN).
|
256 |
* LDSR is an advanced yet slow upscaler, its model and config can be [found here](https://huggingface.co/hollowstrawberry/upscalers-backup/tree/main/LDSR) and both must be placed in `stable-diffusion-webui/models/LDSR`.
|
@@ -346,7 +346,7 @@ Scripts can be found at the bottom of your generation parameters in txt2img or i
|
|
346 |
|
347 |
ControlNet is an extremely powerful recent technology for Stable Diffusion. It lets you analyze information about any previously existing image and use it to guide the generation of your AI images. We'll see what this means in a moment.
|
348 |
|
349 |
-
If you're using the
|
350 |
|
351 |
I will demonstrate how ControlNet may be used. For this I chose a popular image online as our "sample image". It's not necessary for you to follow along, but you can download the images and put them in the **PNG Info** tab to view their generation data.
|
352 |
|
@@ -418,7 +418,7 @@ There are also alternative **diff** versions of each ControlNet model, which pro
|
|
418 |
|
419 |
To train a [Lora â–²](#lora) yourself is an achievement. It's certainly doable, but there are many variables involved, and a lot of work depending on your workflow. It's somewhere between an art and a science.
|
420 |
|
421 |
-
You can do it on your own computer if you have at least 8 GB of VRAM. However, I will be using a Google
|
422 |
|
423 |
Here are some classic resources if you want to read about the topic in depth. Rentry may be blocked by your internet provider, in which case you may use a VPN or try putting it through [Google Translate](https://translate.google.cl/?op=websites).
|
424 |
* [Lora Training on Rentry](https://rentry.org/lora_train)
|
@@ -428,11 +428,11 @@ Here are some classic resources if you want to read about the topic in depth. Re
|
|
428 |
|
429 |
With those way smarter resources out of the way, I'll try to produce a simple guide for you to make your very own Lora for a character, concept, or artstyle.
|
430 |
|
431 |
-
<a name="
|
432 |
|
433 |
-
1. We will be using [THIS
|
434 |
|
435 |
-
1. Click the play button on *🇦 Mount your google drive* and give it access when it asks you to. Do the same for *🇧 Installation*. Proceed to the next step while it installs.
|
436 |
|
437 |
1. Scroll down to *🇨 Settings* but don't run it yet. Here in **Setup** you may give any name you want to your project. You may also change the base model for training, but we'll be using AnythingV3_fp16 as it is the foundation of all anime models. If you want to train with photographs you may copy the link to the [base SD 1.5 model](https://huggingface.co/runwayml/stable-diffusion-v1-5/resolve/main/v1-5-pruned-emaonly.safetensors) or the realistic model you wish to use in the end (such as [Deliberate](https://civitai.com/api/download/models/15236)). In that case remember to change the `model_type` to safetensors.
|
438 |
|
@@ -474,9 +474,9 @@ With those way smarter resources out of the way, I'll try to produce a simple gu
|
|
474 |
|
475 |
Look at that, it gets more detailed over time! The last image is without any Lora for comparison. This was a successful character Lora, at least at first glance. You would need to test different seeds, prompts and scenarios to be sure.
|
476 |
|
477 |
-
It is common that your Lora "fries" or distorts your images when used at high weights such as 1, specially if it's overcooked. A weight of 0.5 to 0.8 is acceptable here, you may need to tweak the learning rate and network dim for this, or other variables not found in this
|
478 |
|
479 |
-
After getting used to making Loras, and hopefully interacting with various resources and the community, you will be ready to use a different method including the [advanced all-in-one
|
480 |
|
481 |
* **Additional tips** <a name="trainchars"></a>[â–²](#index)
|
482 |
|
|
|
55 |
|
56 |
|
57 |
|
58 |
+
# Google Colab <a name="colab"></a>[â–²](#index)
|
59 |
|
60 |
+
The easiest way to use Stable Diffusion is through Google Colab. It borrows Google's computers to use AI, with variable time limitations, usually a few hours every day. You will need at least one Google account and we will be using Google Drive to store your settings and resulting images.
|
61 |
|
62 |
If you instead want to run it on your own computer, [scroll down â–¼](#install).
|
63 |
|
|
|
77 |
|
78 |
# Local Installation (Windows + Nvidia) <a name="install"></a>[â–²](#index)
|
79 |
|
80 |
+
To run Stable Diffusion on your own computer you'll need at least 16 GB of RAM and 4 GB of VRAM (preferably 8). I will only cover the case where you are running Windows 10/11 and using an NVIDIA graphics card series 16XX, 20XX or 30XX (though 10XX also work). My apologies to AMD, Linux, and Mac users, but their cases are harder to cover. If you don't meet the hardware requirements, you can just proceed with the Google Colab method [above â–²](#colab).
|
81 |
|
82 |
1. Get the latest release from [this page](https://github.com/EmpireMediaScience/A1111-Web-UI-Installer/releases).
|
83 |
|
|
|
108 |
|
109 |
![Top](images/top.png)
|
110 |
|
111 |
+
Here you can select your checkpoint and VAE. We will go over what these are and how you can get some. The colab has additional settings here too, you should ignore them for now.
|
112 |
|
113 |
1. **Models** <a name="model"></a>[â–²](#index)
|
114 |
|
|
|
119 |
* For photorealism go with [Deliberate](https://civitai.com/models/4823/deliberate). It can do almost anything, but specially photographs. Very intricate results.
|
120 |
* The [Uber Realistic Porn Merge](https://civitai.com/models/2661/uber-realistic-porn-merge-urpm) is self-explanatory.
|
121 |
|
122 |
+
If you're using the colab in this guide, copy the **direct download link to the file** and paste it in the text box labeled `custom_urls`. Multiple links are separated by commas.
|
123 |
|
124 |
If you're running the program locally, the models normally go into the `stable-diffusion-webui/models/Stable-diffusion` folder.
|
125 |
|
|
|
131 |
|
132 |
Most checkpoints don't come with a VAE built in. The VAE is a small separate model, which "converts your image into human format". Without it, you'll get faded colors and ugly eyes, among other things.
|
133 |
|
134 |
+
If you're using the colab in this guide, you should already have the below VAEs, as I told you to select them before running.
|
135 |
|
136 |
There are practically only 3 different VAEs in circulation:
|
137 |
* [anything vae](https://huggingface.co/WarriorMama777/OrangeMixs/resolve/main/VAEs/orangemix.vae.pt), also known as the orangemix vae. All anime models use this.
|
|
|
159 |
* `EasyNegative, worst quality, low quality, normal quality, child, painting, drawing, sketch, cartoon, anime, render, 3d, blurry, deformed, disfigured, morbid, mutated, bad anatomy, bad art`
|
160 |
|
161 |
* **EasyNegative:** <a name="promptneg"></a>The negative prompts above use EasyNegative, which is an *embedding* or "magic word" that encodes many bad things to make your images better. Otherwise you'd have to use a huge negative prompt.
|
162 |
+
* If you're using the colab in this guide you already have this installed. Otherwise, you will have to [download this tiny file](https://huggingface.co/datasets/gsdf/EasyNegative/resolve/main/EasyNegative.safetensors), put it in your `stable-diffusion-webui/embeddings` folder, then go to the bottom of your WebUI page and click *Reload UI*. It will then work when you type that word.
|
163 |
|
164 |
A comparison with and without these negative prompts including EasyNegative can be seen [further down â–¼](#matrixneg).
|
165 |
|
|
|
219 |
|
220 |
![Extensions](images/extensions.png)
|
221 |
|
222 |
+
Here are some useful extensions. If you're using the colab in this guide you already have most of these, otherwise I hugely recommend you manually add the first 2:
|
223 |
* [Image Browser (fixed fork)](https://github.com/aka7774/sd_images_browser) - This will let you browse your past generated images very efficiently, as well as directly sending their prompts and parameters back to txt2img, img2img, etc.
|
224 |
* [TagComplete](https://github.com/DominikDoom/a1111-sd-webui-tagcomplete) - Absolutely essential for anime art. It will show you the matching booru tags as you type. Anime models work via booru tags, and prompts without them usually don't work, so knowing them is godmode. Not all tags will work well in all models though, specially if they're rare.
|
225 |
* [ControlNet](https://github.com/Mikubill/sd-webui-controlnet) - A huge extension deserving of [its own guide â–¼](#controlnet). It lets you analyze any image and use it as an referene for your own image. Practically speaking, it can create any pose or environment you want.
|
|
|
236 |
|
237 |
Loras can represent a character, an artstyle, poses, clothes, or even a human face (though I do not endorse this). Checkpoints are usually capable enough for general work, but when it comes to specific details with little existing examples, you'll need a Lora. They can be downloaded from [civitai](https://civitai.com) or [elsewhere (NSFW)](https://gitgud.io/gayshit/makesomefuckingporn#lora-list) and are 144 MB by default, but they can go as low as 1 MB. Bigger Loras are not always better. They come in `.safetensors` format, same as most checkpoints.
|
238 |
|
239 |
+
Place your Lora files in the `stable-diffusion-webui/models/Lora` folder, or if you're using the colab in this guide paste the direct download link into the `custom_urls` text box. Then, look for the 🎴 *Show extra networks* button below the big orange Generate button. It will open a new section either directly below or at the very bottom. Click on the Lora tab and press the **Refresh** button to scan for new Loras. When you click a Lora in that menu it will get added to your prompt, looking like this: `<lora:filename:1>`. The start is always the same. The filename will be the exact filename in your system without the `.safetensors` extension. Finally, the number is the weight, like we saw [earlier ▲](#promptweight). Most Loras work between 0.5 and 1 weight, and too high values might "fry" your image, specially if using multiple Loras at the same time.
|
240 |
|
241 |
![Extra Networks](images/extranetworks.png)
|
242 |
|
|
|
250 |
|
251 |
You can download additional upscalers and put them in your `stable-diffusion-webui/models/ESRGAN` folder. They will then be available in Hires fix, Ultimate Upscaler, and Extras.
|
252 |
|
253 |
+
The colab in this guide comes with several of them, including **Remacri**, which is a great all-around upscaler for all sorts of images.
|
254 |
|
255 |
* A few notable ones can be [found here](https://huggingface.co/hollowstrawberry/upscalers-backup/tree/main/ESRGAN).
|
256 |
* LDSR is an advanced yet slow upscaler, its model and config can be [found here](https://huggingface.co/hollowstrawberry/upscalers-backup/tree/main/LDSR) and both must be placed in `stable-diffusion-webui/models/LDSR`.
|
|
|
346 |
|
347 |
ControlNet is an extremely powerful recent technology for Stable Diffusion. It lets you analyze information about any previously existing image and use it to guide the generation of your AI images. We'll see what this means in a moment.
|
348 |
|
349 |
+
If you're using the colab in this guide, you should enable the `all_control_models` option. Otherwise, you should first install the ControlNet [extension â–²](#extensions), then go [here](https://civitai.com/models/9251/controlnet-pre-trained-models) to download some models which you'll need to place in `stable-diffusion-webui/extensions/sd-webui-controlnet/models`. I recommend at least Canny, Depth, Openpose and Scribble, which I will show here.
|
350 |
|
351 |
I will demonstrate how ControlNet may be used. For this I chose a popular image online as our "sample image". It's not necessary for you to follow along, but you can download the images and put them in the **PNG Info** tab to view their generation data.
|
352 |
|
|
|
418 |
|
419 |
To train a [Lora â–²](#lora) yourself is an achievement. It's certainly doable, but there are many variables involved, and a lot of work depending on your workflow. It's somewhere between an art and a science.
|
420 |
|
421 |
+
You can do it on your own computer if you have at least 8 GB of VRAM. However, I will be using a Google Colab document for educational purposes.
|
422 |
|
423 |
Here are some classic resources if you want to read about the topic in depth. Rentry may be blocked by your internet provider, in which case you may use a VPN or try putting it through [Google Translate](https://translate.google.cl/?op=websites).
|
424 |
* [Lora Training on Rentry](https://rentry.org/lora_train)
|
|
|
428 |
|
429 |
With those way smarter resources out of the way, I'll try to produce a simple guide for you to make your very own Lora for a character, concept, or artstyle.
|
430 |
|
431 |
+
<a name="traincolab"></a>![Trainer colab](images/trainercollab.png)
|
432 |
|
433 |
+
1. We will be using [THIS COLAB DOCUMENT](https://colab.research.google.com/drive/1zEm1DlzMfLQe19ly1bAob6Kd22BAl3qL?usp=sharing). You can copy it into your own Google Drive if you want.
|
434 |
|
435 |
+
1. Click the play button on *🇦 Mount your google drive* and give it access when it asks you to. Do the same for *🇧 Installation*. Proceed to the next step while it installs on Google's servers.
|
436 |
|
437 |
1. Scroll down to *🇨 Settings* but don't run it yet. Here in **Setup** you may give any name you want to your project. You may also change the base model for training, but we'll be using AnythingV3_fp16 as it is the foundation of all anime models. If you want to train with photographs you may copy the link to the [base SD 1.5 model](https://huggingface.co/runwayml/stable-diffusion-v1-5/resolve/main/v1-5-pruned-emaonly.safetensors) or the realistic model you wish to use in the end (such as [Deliberate](https://civitai.com/api/download/models/15236)). In that case remember to change the `model_type` to safetensors.
|
438 |
|
|
|
474 |
|
475 |
Look at that, it gets more detailed over time! The last image is without any Lora for comparison. This was a successful character Lora, at least at first glance. You would need to test different seeds, prompts and scenarios to be sure.
|
476 |
|
477 |
+
It is common that your Lora "fries" or distorts your images when used at high weights such as 1, specially if it's overcooked. A weight of 0.5 to 0.8 is acceptable here, you may need to tweak the learning rate and network dim for this, or other variables not found in this colab. If you're reading this and know the magic sauce, let us know.
|
478 |
|
479 |
+
After getting used to making Loras, and hopefully interacting with various resources and the community, you will be ready to use a different method including the [advanced all-in-one colab by kohya](https://colab.research.google.com/github/Linaqruf/kohya-trainer/blob/main/kohya-LoRA-dreambooth.ipynb). Good luck.
|
480 |
|
481 |
* **Additional tips** <a name="trainchars"></a>[â–²](#index)
|
482 |
|