Tricks that sometimes help
Why it matters GAN papers and blog posts list dozens of tricks. Most were shown on one setup, often with one seed. Here you test four of them on the same collapse, over 8 recorded seeds, and report what you measured, not what you read.
The four tricks, each applied to the collapsing GAN with 8 noise numbers:
- Label smoothing: D is told real samples are 0.8 real, not 1, so it never becomes too sure.
- Instance noise: a little noise is added to every point D sees, shrinking to 0 over training, so real and fake overlap early on.
- Spectral norm: every D weight matrix is divided by its largest stretch, so D cannot react too sharply to small changes.
- Learning-rate balance: G slower than D (g_lr 2.5e-4) or faster (g_lr 3e-3).
Use the spirit of Module 9: a difference only counts when it is clearly bigger than the noise between seeds. Compare each trick with the plain run seed by seed (same seed, paired):
- helped: mean blobs at least 1 higher AND more blobs on at least 6 of the 8 seeds;
- hurt: mean blobs at least 1 lower and fewer on at least 6 seeds, OR mean quality at least 0.25 lower and lower on at least 6 seeds (a generator that keeps its blobs but scatters its points has not been helped);
- otherwise no clear change.
Words and symbols in this lesson
- GAN
- generative adversarial network: a generator and a discriminator trained against each other
- generator (G)
- the network that turns noise into a sample
- blobs covered, quality
- the ring judge: blobs with more than 2% of the points, and the share of points on a blob
- label smoothing
- training D on real labels of 0.8 instead of 1
- instance noise
- noise added to everything D sees, shrinking to 0 over training
- spectral norm
- dividing each D weight matrix by its largest stretch
Write verdict(trick) returning "helped", "hurt" or "no clear change" from the recorded runs. Then fill report with your verdict for each of the five trick runs.
Spotted a problem in this lesson? Report it · Ask about this lesson
This exercise opens soon
Finish the lesson before it and this one unlocks. The whole course is free, no account needed, and you can read this lesson right now.
This browser is blocking cookies for learn.floati.life, so lessons you finished here cannot open in order. Allow cookies for this site and reload.