Syllabus Lesson 193 of 246 · Module 18: Learning without labels
Module 18: Learning without labels

Sorting photos nobody labelled

Why it matters Labels are expensive: someone has to look at every picture. Pictures without labels are nearly free. If a network could learn useful features from unlabelled pictures alone, every model could start from millions of examples instead of thousands. That is self-supervised learning, and it is how today's large models are pretrained.

The pictures in this module are tiny 16 x 16 photos of six kinds of object (ball, box, ring, tent, cross, moon) in cluttered scenes: random backgrounds, light from any side, shadows, things in front, blur and sensor noise. They are generated, not downloaded, and built to behave like real photos: the course measured three statistics that every natural photo shares (how detail falls off with scale, how sparse the edges are, how alike neighbouring pixels are), and these photos sit inside the range of real photo crops on all three. photos.stats(X) prints them.

A quick test of any set of features: for each photo, find its 5 nearest neighbours by those features and count how many show the same kind of object. Random guessing gets 1 in 6.

Three kinds of features:

  • the raw pixels;
  • a CNN with random weights, never trained;
  • the same CNN after SimCLR pretraining on 4000 photos, with no labels at all (the course trained it once; you will see how in lesson 5).

Predict first: which puts photos of the same object closest together?

Words and symbols in this lesson
self-supervised learning
learning features from data with no labels, by a task the data sets itself
SimCLR
self-supervised training with two views of each picture and InfoNCE (Chen et al. 2020)
Your turn

Set my_guess to "raw", "random" or "simclr". Write agreement(F, labels, k): standardise each feature, scale each row to length 1, and return the share of the k nearest neighbours (by cosine similarity, a photo is not its own neighbour) that have the same label. Fill scores for the three kinds of features.

Spotted a problem in this lesson? Report it · Ask about this lesson

This exercise opens soon

Finish the lesson before it and this one unlocks. The whole course is free, no account needed, and you can read this lesson right now.