professional headshot photo of Mr Palumbo

Building This Nature Does Not Exist

thisnaturedoesnotexist.com shows you a landscape that was never photographed. Every image on it was made by a diffusion model I wrote myself, from an empty file, in PyTorch - no pretrained weights and no hosted API anywhere in the stack.

Sixteen images from the live gallery. None of these places exists; each one started as random noise.

This post is the short version of how it got built: a laptop, a rented GPU, a crash, a few bugs, and the decisions that made each step cheaper or faster. If you want the math and the architecture, that is in the companion post, Building a Diffusion Model From Scratch.

Toy2D spiral of points, tiny MLP
→
Laptop (M1)32px → 64px, ~6k photos
→
Rented RTX 4090128px, ~90k photos, ~$8
→
Static sitepre-generated gallery, no server GPU
~48M
parameters
~90k
training photos
~$8
GPU cost
0
pretrained weights
01 · The idea

Why nature that doesn't exist

The idea borrows from thispersondoesnotexist.com: one page, one job, a fresh generated image every time you look. I picked landscapes instead of faces.

The real goal was to understand image generation all the way down. A lot of "AI image" projects wrap someone else's model. I wanted to write every layer myself, train it on hardware I paid for by the hour, and ship it - then write up what happened honestly, including the parts that went wrong.

02 · Phase 0

A spiral made of noise

Before touching a single image, I built a toy. The whole dataset was a scatter of 2D points in a spiral shape, and the network was a tiny multi-layer perceptron. The model learned to start from random dots and walk them back into the spiral.

That sounds trivial, and it was meant to be. With no images and no complex network, there was nowhere for a bug to hide. Once the toy worked, I knew the core training and sampling loop was right, and I could scale it up to pixels with confidence.

Six panels: the real spiral of points, pure noise at t=199, then t=150, 100 and 50 as the points gather, and the generated spiral at t=0
The toy model at work. Left: the real data, a spiral of 2D points. Then the reverse walk, from pure noise at t=199 to a generated spiral at t=0. It is the same math that later makes landscapes.
03 · Phase 1

Training on a laptop

The first real models trained on my M1 laptop: about 6,000 landscape photos from the monet2photo dataset, first at 32 pixels, then at 64. The results were soft coloured blobs that were starting to look like horizons. Good enough to prove the pipeline end to end.

32px · epoch 39
32px · epoch 39
64px · epoch 59
64px · epoch 59
The best the laptop could do: real colour and rough horizons, but no detail.

The 32-to-64 handoff caught a bug that would have cost real money later. When I loaded the 32px weights and asked for "60 epochs", my --epochs flag was being read as an absolute epoch number, not "run this many more". A run I believed was 60 epochs would have silently trained only 20. I caught it before it wasted a run and fixed it.

The laptop also had a blind spot. Mixed precision (AMP), which roughly halves memory and speeds up training on NVIDIA GPUs, could not be tested on the M1 at all. I wrote the code anyway. The first confirmation that it worked was a single line printed on the rented cloud machine: device: cuda amp: True.

04 · Phase 2

Renting a GPU for about $8

The real run trained a bigger model at 128 pixels on about 90,000 landscapes from the LHQ dataset. That needed a real GPU, so I rented one on RunPod. Before picking, I worked out what each option would actually cost.

The formula that matters Total cost = wall-clock time × hourly rate. The cheapest card per hour is almost never the cheapest overall, and the fastest card is wasted on a small model.

I measured one real number - the RTX 4090 took 433 seconds per epoch - and estimated the rest from it, so they are ballpark:

A40$0.44/hr~$8.2RTX 4090$0.69/hr~$8.3RTX 5090$0.99/hr~$8.5RTX 3090$0.46/hr~$9.2L4$0.39/hr~$11.7A100$1.39/hr~$13.5H100$2.99/hr~$24B200$5.89/hr~$44
Estimated total cost of the 100-epoch, 128px run on each card. Only the RTX 4090 row is measured (433 s per epoch); the rest are extrapolated from it. The L4 is the cheapest per hour and one of the most expensive in total.

The big data-center cards were money on fire for this job: a 48M-parameter model cannot fill their memory or keep their cores busy. At 433 seconds per epoch, the 4090 needed about 12 hours for 100 epochs.

The run did not go cleanly. At epoch 32 it crashed: one worker failed to read one image off the network storage volume (OSError: [Errno 6]). That was not a code or data bug - all 90,000 files had been read cleanly 31 times already. It was a momentary storage hiccup. I made the data loader retry a failed read and fall back to a neighbouring image, then resumed from the epoch-31 checkpoint. Because I saved a checkpoint every epoch, the crash cost about seven minutes instead of the whole run.

×epoch 32: one read fails, run diesresume from epoch-31 checkpoint: ~7 min lost0100 epochs| = checkpoint saved
Because every epoch wrote a checkpoint, a crash a third of the way through cost minutes, not the whole run.

The jump in quality from the laptop to the cloud was huge. It is tempting to credit the higher resolution, but most of it came from the data: 15× more images, and cleaner ones.

05 · Phases 3-4

Shipping it

My first plan was to run the model live in the visitor's browser, with no server GPU at all. I exported it to ONNX (about 95-190MB) and ran it through ONNX Runtime Web on WebGPU. It worked: 18.00 seconds per image on average on a laptop.

That number decided it. Eighteen seconds is too long to be the default "click and see" experience, and phones and browsers without WebGPU would be slower or unable to run it at all. thispersondoesnotexist.com does not run its model live either - it serves pre-generated images. I first planned a hybrid with live generation as an opt-in, then in October dropped live generation entirely. Today every image on the site is pre-generated by the model and ships with the static site, so it loads instantly.

06 · Retrospective

What I would do differently

Fill the GPU I already paid for. I chose the 4090 carefully, then set the batch size to 32, which barely touched its 24GB of memory. At a batch of 96-128 the same card would have finished faster for the same hourly price. The biggest lever was never the GPU.

Trust the samples, not the loss. The training loss fell from about 0.043 to 0.026 in a few epochs and then barely moved, while the images kept getting much better. On a generative model, the sample grid is the real progress signal.

0.000.100.200.2540.0670.0590.053training epoch (0 to 39) →
Sample grid at epoch 0
epoch 0 · loss 0.254
Sample grid at epoch 10
epoch 10 · loss 0.067
Sample grid at epoch 20
epoch 20 · loss 0.059
Sample grid at epoch 39
epoch 39 · loss 0.053
Measured on the 32px laptop run. Between epoch 10 and epoch 39 the loss barely moves (0.067 to 0.053), while the samples go from static to landscapes.

Do not trust the first read of a result. When training finished, the sample images had speckles and solid flat-colour tiles. I assumed the model was undertrained: 100 epochs is about 281,000 training steps, and the original DDPM paper used 800,000+ for images a quarter the size. I was ready to rent more GPU time.

64px laptop run, epoch 59
laptop · 64px · epoch 59
128px cloud run, epoch 99 training preview
cloud · 128px · epoch 99 preview
Same 128px model with the fixed sampler settings
same model · fixed sampler
The epoch 99 preview is what I saw when training finished, and it looked like a training problem. The last grid is the same trained model with the sampler fixed. Nothing about the model changed.

Then I tested the full, slow 1000-step sampler as a reference, and it came back worse - tiles of pure red, white, black and cyan, the corners of colour space where numbers blow up. That code path had never run on a real trained model. It lacked a safety clamp the fast sampler already had, so small errors compounded over 1000 steps. I fixed it, re-ran the same seed, and got real landscapes: a coastal cliff with reflections, a glacier lake, sunsets. The model was fine. The sampler was the dominant problem.

before: 1000-step sampler, no clamp
before: 1000-step sampler, no clamp
after: same seed, clamp added
after: same seed, clamp added
Same checkpoint, same starting noise. Without the clamp, the solid tiles are the corners of RGB space - values that blew up. With it, real landscapes. The sampler fix is the only change.

It happened twice. The fast sampler still showed some "confetti" artifacts, so I tested more steps: 50, 75 and 100 changed almost nothing. Adding a little randomness back into each step fixed it entirely, at 3.6 seconds per image instead of about 29 for the slow sampler. Both quality problems on this project turned out to be the sampler, not the model - and both were found by testing, not by assuming a component worked because it looked right on paper.

deterministic DDIM, 50 steps
deterministic DDIM, 50 steps
DDIM, 100 steps, eta=0.5
DDIM, 100 steps, eta=0.5
The fast deterministic sampler on the final checkpoint gives speckle and confetti tiles. With 100 steps and eta=0.5, the setting the live gallery uses, they are gone; these 16 tiles are from that gallery.

Where it ended up

A model I wrote from scratch, trained for about $8 of rented GPU time, serving convincing landscapes on thisnaturedoesnotexist.com. The most useful part was not the result. It was the habits the bugs taught: checkpoint every epoch, make long jobs survive a bad read, measure cost as time × rate, and test every code path on real output before you trust it.

For the full technical version - the U-Net, the noise schedule, the sampler math, and the code for each fix - read Building a Diffusion Model From Scratch.