Syllabus Lesson 178 of 246 · Module 16: Two networks in a game (GANs, optional)
Module 16: Two networks in a game (GANs, optional)

Mode collapse, on purpose

Why it matters The most common GAN failure looks like success: every sample is sharp and convincing, but they are all nearly the same. It is called mode collapse (each blob is a "mode" of the data). You cannot fix what you cannot make happen, so here you make it happen.

Why it happens: G only has to fool today's D. If one blob fools D best, G puts everything there. D learns that blob is suspicious, G jumps to another one, and the two can chase each other round the ring without G ever spreading out.

ganlab.train(...) trains the same GAN as the last lesson with any of these settings: noise (how many noise numbers G gets, 2 by default), d_steps (D's steps per G step, 1), g_lr and d_lr (1e-3 each), loss, smooth, instance_noise, spectral. The recorded runs, ganlab.recorded(), show some settings over 8 seeds.

Collapse means few blobs AND sharp points. A generator that scatters points everywhere covers few blobs too, but that is a different failure: it never learned the blobs at all.

Words and symbols in this lesson
GAN
generative adversarial network: a generator and a discriminator trained against each other
generator (G)
the network that turns noise into a sample
mode collapse
the generator makes only a few kinds of sample, each convincing
blobs covered, quality
the ring judge: blobs with more than 2% of the points, and the share of points on a blob
Your turn

Find settings that collapse the GAN on seed 0: at most 3 blobs covered, with quality at least 0.6. Put them in settings. To keep each try short, this lesson trains for 1000 steps at a learning rate of 0.002 (set for you; the collapse it shows is the same as at 2000 steps). Change only the GAN's own settings (noise, d_steps up to 3, loss, smooth, instance_noise, spectral); the seed, steps and learning rates stay as given. Then set looks_like to the letter that describes collapse.

Spotted a problem in this lesson? Report it · Ask about this lesson

This exercise opens soon

Finish the lesson before it and this one unlocks. The whole course is free, no account needed, and you can read this lesson right now.