Stable Diffusion QR Codes

QR Codes are machine-readable codes that hold text information like URLs. Modern smartphones are able to use their cameras to process the qr codes and pull up a webpage that is encoded within the QR code. QR codes have mostly been a single color or a gradient to some extent to bring style to an individual QR code. However, now we can use AI to really push the boundaries of what a QR code can look like and still be functional.

Automatic1111

Automatic1111 is a broswer based user interface for Stable Diffusion. Stable Diffusion can be used from the command line but having a user interface helps when working with multiple models. We need to download and run Automatic1111 to be able to use this workflow for creating Stable Diffusion QR codes.

Checkpoints

Stable diffusion needs models to be able to generate images. Models that are used are sometimes referred to as checkpoint files which are pre-trained stable diffusion weights created to generated general images or a more specific niche of images. We can train our own model from base models like Stable Diffusion v1.4 or v1.5 but we are going to use the GhostMix model on CivitAi since it will allow us to jump right in to generating images.

CivitAi

CivitAi is a helpful site to find stable diffusion models. This website is great because you can see example images that have been generated with given models. For this workflow I have chosen the GhostMix Model linked below.

Hugging Face

Hugging Face is a community driven website full of machine learning information. For creating QR code images with stable diffusion we will only be using a small portion of Hugging Face. Definitely check out this website to see what they are up to if you are interested in doing more with machine learning.

ControlNet

ControlNet is a neural network structure that allows for adding conditional control to text-to-image diffusion models. To create QR code images with Stable Diffusion we need to download ControlNet in Automatic1111. Paste the ControlNet Git link below into the "Install from URL" form input on the Extensions tab in Automatic1111.

Tile

After the ControlNet extension is installed we need to get the ControlNet tile model from HuggingFace. We only need the control_xxxx_tile.pth file for this workflow. The file will be around 3GB in size.

Workflow

Now that every dependency is installed we can begin to create the QR code images. The below workflow is a good starting point to generate nice styles. Remember that the QR code you input will also have an effect on the output so what you get will not necessarily be the same as the above QR codes showcased.

1. Go to the img2img tab
2. Stable Diffusion checkpoint: ghostmix
3. Enter the prompt for the image
4. Enter the negative prompt for the image
5. Add the QR code you have created to the first image input
6. Sampling method: DPM++ 2M KARRAS
7. Sampling Steps: 50
8. Width: 512
9. Height: 512
10. CFG Scale: 7
11. Denoising strength: 0.75
12. Open the dropdown for the ControlNet extension
13. Add the QR code you have created to the image input
14. Check the "Enable" checkbox
15. Preprocessor: tile_resample
16. Model: control_xxxx_tile (Note: xxxx will be the version of the model you downloaded)
17. Control Weight: 0.75
18. Starting Control Step: 0.23
19. Ending Control Step: 0.9
20. Control Mode: Balanced
21. Click Generate

Tips

QR code Not Working

Not every output will be a working QR code. Some of the generated QR codes will only work at a certain distance when checking with your phone. The smaller the link in the QR code the better off you will be since there will be less pixels needed to make up the QR code. Having a small QR code is not always possible so continue to generate images and you will get one that works after changing the inputs to be more or less like your control.

Seed or No

You will not get the same output unless you keep the same seed. Having the seed set to -1 will use a random seed for each generation. I found it helpful to create random QR code images until I found a theme that I liked and then recycling the seed while changing some of the inputs.

References