Syllabus Lesson 168 of 246 · Module 15: Squeeze, rebuild, generate (autoencoders and VAEs)
Module 15: Squeeze, rebuild, generate (autoencoders and VAEs)

Why an autoencoder cannot make new digits

Why it matters If the decoder turns 16 numbers into a digit, why not pick 16 numbers at random and get a brand new digit? Try it. What you see explains what the rest of this module has to fix.

To say how good generated pictures are, you need a judge. The course's judge measures two things:

  • Closeness: the distance from each picture to the nearest real training digit, averaged. Real test digits score 1.04. Pixels scrambled inside each picture score 3.52.
  • Variety: a digit reader (98% right on real digits) says which digit each picture shows; variety is how evenly the ten digits appear, in bits. 3.32 means all ten equally often, 0 means every picture reads as the same digit.

Why two numbers? Because closeness alone can be fooled. You will see how.

Predict first: when you decode random codes, picked anywhere inside the range the real codes cover, will the pictures score closer to real digits than the average picture of each digit does?

Words and symbols in this lesson
autoencoder
an encoder that squeezes a picture into a few numbers and a decoder that rebuilds it
code (latent)
the few numbers the encoder makes from one picture
closeness, variety
the course judge: distance to the nearest real digit, and bits of spread over the ten digits
Your turn

Set my_prediction to "closer" or "further". Then fill in scores with the closeness of: the real test digits, the ten average digits (each repeated to 500 pictures), and 500 decoded random codes. Finally set reason to the letter of the explanation the numbers support.

Spotted a problem in this lesson? Report it · Ask about this lesson

This exercise opens soon

Finish the lesson before it and this one unlocks. The whole course is free, no account needed, and you can read this lesson right now.