Syllabus Lesson 225 of 246 · Module 21: Looking inside a model
Module 21: Looking inside a model

Reading a feature, and checking the reading

Why it matters A feature gets a name from its top examples. That name is a hypothesis, and most published feature labels are never tested against text the labeller did not see.

Load the course's recorded sparse autoencoder (256 features, l1 0.3, trained for 2,000 steps of 512 vectors, longer than a lesson allows, and the same features on every machine) and pick, for a few strings, the feature most specific to positions where the text ends with that string. Then label it and test the label on held-out text:

  • precision: of the held-out places where the feature fires strongly (above half its largest training value), the share where the text really ends with the string;
  • recall: of the held-out places that end with the string, the share where it fires.

interp.top_contexts(sae, A, X, model, k) shows where feature k fires hardest, as the text up to that character.

Predict first: the feature for "tra" fires hardest on " before tra" and " layers tra" (the start of "train"). Will a "tra" label be right most of the time on new text?

Words and symbols in this lesson
sparse autoencoder
a wide layer trained to rebuild vectors from a few active features at a time
precision, recall (of a label)
of the places a feature fires, the share the label fits; of the places the label fits, the share it fires
Your turn

Set my_guess to "yes" or "no". Write check(k, s) returning (precision, recall) on the held-out vectors, where the feature fires at a place when its value there is above half its largest value on the training vectors (F[:, k].max()). Fill results[s] = check(feature_for(s), s) for " the", "ing" and "tra". Set honest_label to the strings whose label holds (precision at least 0.7) and really to the three-letter ending the "tra" feature fires on most in held-out text.

Spotted a problem in this lesson? Report it · Ask about this lesson

This exercise opens soon

Finish the lesson before it and this one unlocks. The whole course is free, no account needed, and you can read this lesson right now.