Syllabus Lesson 179 of 246 · Module 16: Two networks in a game (GANs, optional)
Module 16: Two networks in a game (GANs, optional)

Tricks that sometimes help

Why it matters GAN papers and blog posts list dozens of tricks. Most were shown on one setup, often with one seed. Here you test four of them on the same collapse, over 8 recorded seeds, and report what you measured, not what you read.

The four tricks, each applied to the collapsing GAN with 8 noise numbers:

  • Label smoothing: D is told real samples are 0.8 real, not 1, so it never becomes too sure.
  • Instance noise: a little noise is added to every point D sees, shrinking to 0 over training, so real and fake overlap early on.
  • Spectral norm: every D weight matrix is divided by its largest stretch, so D cannot react too sharply to small changes.
  • Learning-rate balance: G slower than D (g_lr 2.5e-4) or faster (g_lr 3e-3).

Use the spirit of Module 9: a difference only counts when it is clearly bigger than the noise between seeds. Compare each trick with the plain run seed by seed (same seed, paired):

  • helped: mean blobs at least 1 higher AND more blobs on at least 6 of the 8 seeds;
  • hurt: mean blobs at least 1 lower and fewer on at least 6 seeds, OR mean quality at least 0.25 lower and lower on at least 6 seeds (a generator that keeps its blobs but scatters its points has not been helped);
  • otherwise no clear change.
Words and symbols in this lesson
GAN
generative adversarial network: a generator and a discriminator trained against each other
generator (G)
the network that turns noise into a sample
blobs covered, quality
the ring judge: blobs with more than 2% of the points, and the share of points on a blob
label smoothing
training D on real labels of 0.8 instead of 1
instance noise
noise added to everything D sees, shrinking to 0 over training
spectral norm
dividing each D weight matrix by its largest stretch
Your turn

Write verdict(trick) returning "helped", "hurt" or "no clear change" from the recorded runs. Then fill report with your verdict for each of the five trick runs.

Spotted a problem in this lesson? Report it · Ask about this lesson

This exercise opens soon

Finish the lesson before it and this one unlocks. The whole course is free, no account needed, and you can read this lesson right now.