Before beginning your character sprites, we need to setup our Stable Diffusion environment first. Firstly, we need to see whether your system is capable of running Stable Diffusion.
¶ System Requirements
There is no concerete system requirements to run Stable Diffusion, except having a GPU with enough VRAM.
While the Stable Diffusion WebUI reports that you can run image generation on 4 GB VRAM, our suggestion is to use somewhere between 6-8 GB or more, depending on the resoulution your model is trying to be. By increasing the size from 512x512 to something like 720x960, you are adding more data which in turn, requires more VRAM to create the sprite that you want. Adjust resolution sizes to be what you want your sprite to be and close to your total VRAM size or a adequate size of your choosing.
¶ Installing Stable Diffusion
Installing Stable Diffusion is pretty easy. Just make sure to install the following on your system before proceeding.
- Git
- Python 3.10
Make sure you install Python 3.10 and not Python 3.11 or later.
You can have Python 3.10 installed on a system with an older or newer version of Python, however you will have to use
py -3.10instead ofpythonorpython3in your Terminal/Command Prompt session.
For Linux users, you may also need to install
python3-venvonto your system to add Python virtual environments.
¶ Installation
- Install Git by following the steps on this page.
- Download and install Python 3.10
- (For Windows 10/11) Go to the Start Menu and search for Manage app execution aliases. Click on it and you should see the following.
Turn off the settings for python.exe and python3.exe.
- Open a Terminal/Command Prompt session and copy the following line:
git clone https://github.com/AUTOMATIC1111/stable-diffusion-webui.git && cd stable-diffusion-webui
- (Recommended for Linux) Install
tcmalloconto your system.
sudo apt install libgoogle-perftools-dev
- Go to Civitai and download a AI model to use.
For this guide, I will be using AnyLoRA - Checkpoint with the option
bakedVae (blessed) fp16 NOT-PRUNEDselected.
-
Save the model you downloaded to
[Stable Diffusion WebUI Folder]/models/Stable-diffusion. -
Download the following Embeddings and save them to
[Stable Diffusion WebUI Folder]/embeddings.
- EasyNegative Civitai (EasyNegative)
- badhandsv5-neg HuggingFace (badhandsv5-neg.pt)
- DeepNegative Civitai (v1_75T)
Step 9 is if you are making a sprite for a character from a franchise.
OCs may skip this step or search a LoRA for specific attributes (Foxgirl, Lamia, etc.)
- (Optional) Search Civitai for a LoRA for your character, and save it to
[Stable Diffusion WebUI Folder]/models/Lora.
A LoRA is simply a minature model that has been trained specifically on one specific aspect of content. In terms of Stable Diffusion, LoRAs are models that have been trained based off art references of a certain character.
For the rest of this guide, we will be using this Bronya Rand LoRA on Civitai.
Do not fret if you cannot find a LoRA of your character. You can still make sprites for your character by specifying how the character looks like and what they wear.
Now that you installed Stable Diffusion and gotten a AI model (and LoRA) downloaded, go back to your Terminal/Command Prompt session and launch Stable Diffusion by copying the following lines, depending on your operating system:
Windows
.\webui.bat
Linux
./webui.sh --xformers
Let Stable Diffusion download and install what it needs. Once it finished setting up, you should see a link in the Terminal/Command Prompt. Go to the link posted and you should be welcomed to the Stable Diffusion WebUI itself.
¶ Post-Installation!
- Go to Settings > Stable Diffusion and set Clip skip to 2.
- Go to Extensions > Available and click on
Load from:. You should see this afterwards:
- Install the following extensions:
- a1111-sd-webui-tagcomplete
This extension allows you to use tags commonly used in booru sites (Danbooru, Gelbooru) in your prompt/blacklist.
- adetailer
This extension allows you to give your created image a second (or third) passthrough on specific features you wish to enhance further.
- stable-diffusion-webui-rembg
This extension removes the background of images, producing a transparent image.
- Go back to Installed and click on Apply and restart UI and wait for Stable Diffusion to restart.
¶ Overview
Once Stable Diffusion restarts, you are now ready to go with creating your character sprites! Congratulations! In the next section, we will cover using Stable Diffusion to generate our character sprites using the settings presented to us in txt2img and img2img and in-painting.
Next Step