Diffusion models (DDPM)

The generative-models overview sketched diffusion as: add noise to an image step by step, then train a network to undo it. This lesson is the DDPM (Denoising Diffusion Probabilistic Models) math underneath — and the surprise is how simple the real training objective is. That simplicity is the whole reason diffusion overtook GANs and VAEs.

The forward process has a closed form

The forward process is fixed, not learned: over T steps (e.g. 1000), add a little Gaussian noise each step until the image is pure noise. The crucial fact is that those per-step Gaussians compose, so you can jump directly to any step t in one shot — no iteration:

x_t = √(ᾱ_t) · x_0  +  √(1 − ᾱ_t) · ε        ε ~ N(0,1),   ᾱ_t = ∏(1 − β_i)
      \__________/     \____________/
       signal coeff      noise coeff

As t grows, ᾱ_t decays from ~1 to ~0, so the signal coefficient fades and the noise coefficient takes over — clean image to pure noise:

import numpy as np

rng = np.random.default_rng(0)

x0 = 1.0
betas = np.linspace(1e-4, 0.02, 1000)
alpha_bar = np.cumprod(1 - betas)              # composes the per-step noise schedule

for t in [0, 100, 500, 999]:
    ab = alpha_bar[t]; eps = rng.standard_normal()
    xt = np.sqrt(ab) * x0 + np.sqrt(1 - ab) * eps
    print(f"t={t:>3}: alpha_bar={ab:.3f}  x_t={xt:+.2f}  (signal x{np.sqrt(ab):.2f}, noise x{np.sqrt(1-ab):.2f})")

t=  0: alpha_bar=1.000  x_t=+1.00  (signal x1.00, noise x0.01)
t=100: alpha_bar=0.895  x_t=+0.90  (signal x0.95, noise x0.32)
t=500: alpha_bar=0.078  x_t=+0.89  (signal x0.28, noise x0.96)
t=999: alpha_bar=0.000  x_t=+0.11  (signal x0.01, noise x1.00)

By t=999 the signal coefficient is ~0.01 and the noise coefficient ~1.0 — the original is gone. The one-step formula is what makes training efficient: you can sample any (x_t, t) pair instantly without simulating the chain.

The objective: just predict the noise

Here’s the elegant part. The network (a U-Net with attention) doesn’t predict the image or a distribution — it predicts the noise ε that was added at step t. The entire loss is a plain MSE between the true noise and the predicted noise:

loss = ‖ ε − ε_θ(x_t, t) ‖²

Training: take a clean image, pick a random t, noise it with the closed form, and ask the network to recover the ε. Each step is an easy regression problem — remove a little noise, not paint an image from nothing — and the difficulty is spread across all T steps. There’s no adversarial game (so no GAN instability) and the model sees data at every noise level, giving it broad coverage of the distribution.

Sampling: slow but worth it

To generate, start from pure noise x_T ~ N(0,1) and apply the denoiser T times, each step removing a little noise, until a clean image emerges. The model never paints in one shot — it refines iteratively. The cost is speed: ~1000 forward passes per image. Practical samplers (DDIM, and distilled few-step models like LCM) cut that to 20–50 — or even 1–4 — steps with modest quality loss.

In one breath

• DDPM has a fixed forward process that adds Gaussian noise over T steps; the per-step Gaussians compose, giving a closed form x_t = √ᾱ_t·x_0 + √(1−ᾱ_t)·ε that jumps to any step in one shot.
• As t grows, ᾱ_t → 0, so the signal fades and noise takes over (the demo: signal ×1.0 → ×0.01).
• The only learned part is a U-Net that predicts the added noise ε with a plain MSE loss ‖ε − ε_θ(x_t,t)‖² — each step an easy regression, difficulty spread over T steps.
• Sampling starts from noise and applies the denoiser T times (slow, ~1000 passes; cut to 20–50 or fewer with DDIM/LCM).
• Diffusion wins on stability (no minimax), no mode collapse, and quality/diversity; its slowness drove latent diffusion, flow matching, and distillation.

Quick check

Next

Diffusion’s slowness is solved in latent diffusion (denoise in a VAE latent); both sit in the generative-models overview. The denoiser’s attention is self-attention.