GANs from scratch

The generative-models overview framed a GAN as a generator versus a discriminator. This lesson writes down the game they’re actually playing — the minimax objective, how you train two competing networks without one crushing the other, and why the whole thing is famously delicate.

The minimax objective

A GAN has two networks with opposite goals over one objective:

min_G  max_D   E_x[ log D(x) ]  +  E_z[ log(1 − D(G(z))) ]
               \____________/      \___________________/
               D: score real high   D: score fakes low

The discriminator D maximizes this — push D(x) toward 1 on real data and D(G(z)) toward 0 on fakes. The generator G minimizes it — make D(G(z)) large, i.e. fool D into scoring its fakes as real. It’s a zero-sum game: G’s gain is D’s loss. There’s no fixed “looks realistic” loss written by hand — D is the loss, learned from data and improving as G improves.

Training: alternate, and don’t let G starve

You can’t minimize and maximize at once, so you alternate: take a gradient step to improve D, then a step to improve G, repeat. One subtlety matters in practice. Early on, G is terrible, D rejects its fakes confidently (D(G(z)) ≈ 0), and the log(1 − D(G(z))) term saturates — its gradient vanishes, so G can’t learn. The fix is the non-saturating loss: instead of minimizing log(1 − D(G(z))), G maximizes log D(G(z)). Same goal (fool D), but a strong gradient exactly when G is losing.

The equilibrium

At the game’s optimum, G’s distribution equals the real data distribution, and then D is helpless — every input, real or fake, is equally likely, so the best it can do is output 0.5. As G improves, D’s accuracy decays toward that coin flip:

real_mean, gen_mean = 0.0, 3.0
for step in range(1, 6):
    gen_mean += 0.5 * (real_mean - gen_mean)          # G moves its distribution toward real
    gap = abs(gen_mean - real_mean)
    d_acc = 0.5 + 0.5 * min(1.0, gap / 3.0)           # D: ~1.0 when far apart, 0.5 when matched
    print(f"step {step}: gen_mean={gen_mean:+.2f}  D accuracy={d_acc:.2f}")

step 1: gen_mean=+1.50  D accuracy=0.75
step 2: gen_mean=+0.75  D accuracy=0.62
step 3: gen_mean=+0.38  D accuracy=0.56
step 4: gen_mean=+0.19  D accuracy=0.53
step 5: gen_mean=+0.09  D accuracy=0.52

As the generator’s distribution closes on the real one, the discriminator’s accuracy slides from 0.75 toward 0.5 — the signature that the GAN has converged: D can no longer tell real from fake.

Why GANs are so hard — and the variant family

That clean equilibrium is rarely reached smoothly. The minimax is a saddle point, not a minimum, so training can oscillate instead of converging. If D gets too strong too fast, G’s gradient vanishes (the saturation problem); if G finds one output that reliably fools D, it stops exploring — mode collapse. Stabilizing GANs is a whole literature: WGAN (a smoother Wasserstein loss), spectral normalization, and careful balancing of the two networks.

In one breath

• A GAN plays a minimax game: min_G max_D E[log D(x)] + E[log(1−D(G(z)))] — D scores real high and fakes low; G fools D. D is the learned loss.
• You alternate D and G gradient steps; early on G’s loss saturates, so use the non-saturating form (G maximizes log D(G(z))) for a strong gradient when G is losing.
• At equilibrium, G’s distribution equals the data’s and D outputs 0.5 everywhere — the demo’s D accuracy slides 0.75 → ~0.5 as G matches real.
• GANs are unstable: a saddle point (oscillation), vanishing gradient if D dominates, and mode collapse; fixes include WGAN and spectral normalization.
• The variant family — DCGAN, conditional GAN/CTGAN, pix2pix/CycleGAN, StyleGAN — and the lasting idea that a discriminator is a learned loss reusable beyond images.

Quick check

Next

The GAN’s stable-but-blurry counterpart is VAEs from scratch; both sit in the generative-models overview. Conditional GANs for rare-class oversampling connect to class imbalance.