Before beginning your character sprites, we need to setup our Stable Diffusion environment first. Firstly, we need to see whether your system is capable of running Stable Diffusion.
There is no concerete system requirements to run Stable Diffusion, except having a GPU with enough VRAM.
While the Stable Diffusion WebUI reports that you can run image generation on 4 GB VRAM, our suggestion is to use somewhere between 6-8 GB or more, depending on the resoulution your model is trying to be. By increasing the size from 512x512 to something like 720x960, you are adding more data which in turn, requires more VRAM to create the sprite that you want. Adjust resolution sizes to be what you want your sprite to be and close to your total VRAM size or a adequate size of your choosing.
Installing Stable Diffusion is pretty easy. Just make sure to install the following on your system before proceeding.
Make sure you install Python 3.10 and not Python 3.11 or later.
You can have Python 3.10 installed on a system with an older or newer version of Python, however you will have to use
py -3.10instead of
python3in your Terminal/Command Prompt session.
For Linux users, you may also need to install
python3-venvonto your system to add Python virtual environments.
Turn off the settings for
git clone https://github.com/AUTOMATIC1111/stable-diffusion-webui.git && cd stable-diffusion-webui
tcmalloconto your system.
sudo apt install libgoogle-perftools-dev
For this guide, I will be using AnyLoRA - Checkpoint with the option
bakedVae (blessed) fp16 NOT-PRUNEDselected.
Save the model you downloaded to
[Stable Diffusion WebUI Folder]/models/Stable-diffusion.
Download the following Embeddings and save them to
[Stable Diffusion WebUI Folder]/embeddings.
Step 9 is if you are making a sprite for a character from a franchise.
OCs may skip this step or search a LoRA for specific attributes (Foxgirl, Lamia, etc.)
[Stable Diffusion WebUI Folder]/models/Lora.
A LoRA is simply a minature model that has been trained specifically on one specific aspect of content. In terms of Stable Diffusion, LoRAs are models that have been trained based off art references of a certain character.
For the rest of this guide, we will be using this Bronya Rand LoRA on Civitai.
Do not fret if you cannot find a LoRA of your character. You can still make sprites for your character by specifying how the character looks like and what they wear.
Now that you installed Stable Diffusion and gotten a AI model (and LoRA) downloaded, go back to your Terminal/Command Prompt session and launch Stable Diffusion by copying the following lines, depending on your operating system:
Let Stable Diffusion download and install what it needs. Once it finished setting up, you should see a link in the Terminal/Command Prompt. Go to the link posted and you should be welcomed to the Stable Diffusion WebUI itself.
Load from:. You should see this afterwards:
This extension allows you to use tags commonly used in booru sites (Danbooru, Gelbooru) in your prompt/blacklist.
This extension allows you to give your created image a second (or third) passthrough on specific features you wish to enhance further.
This extension removes the background of images, producing a transparent image.
Once Stable Diffusion restarts, you are now ready to go with creating your character sprites! Congratulations! In the next section, we will cover using Stable Diffusion to generate our character sprites using the settings presented to us in txt2img and img2img and in-painting.