Training GANs That Don't Collapse: Lessons from DCGAN to StyleGAN

Why GANs Fail

1. Mode collapse: Generator outputs same image regardless of noise
2. Training instability: G and D loss oscillate without converging
3. Vanishing gradients: Discriminator becomes too strong

Fix 1: Spectral Normalization

from torch.nn.utils import spectral_norm

self.conv = spectral_norm(nn.Conv2d(in_ch, out_ch, 4, 2, 1))

Fix 2: Label Smoothing for Discriminator

real_labels = torch.ones(B) * 0.9  # not exactly 1.0
fake_labels = torch.zeros(B) + 0.1  # not exactly 0.0

Fix 3: Gradient Penalty (WGAN-GP)

Penalizes the gradient norm of the discriminator — most stable GAN variant for real applications.

Fix 4: Two Time-Scale Update (TTUR)

Use different learning rates: G=0.0001, D=0.0004.