Syllabus Lesson 148 of 156 · Measure the web like a researcher
Measure the web like a researcher

Train a tracker classifier

Why it matters Blocklists are written by people and always lag behind: new trackers appear every week. A classifier learns what trackers behave like and can flag ones no list has named yet. Building a small one yourself shows what such tools can do, and how to judge them honestly.

Think of an experienced doorman. Nobody gave them a list of every troublemaker, but they have learned the signs, and they weigh them together: one sign means little, three together mean a lot. A classifier does the same with numbers.

The data

tracker_dataset(seed) in watched.leg7b_web crawls a simulated web and measures six behaviours for every third party it saw, each scaled from 0 to 1: how many sites it is on, how often it sets cookies, how often it sends tiny images (pixels), how often its URLs carry long ID-like values, how often they carry the page address, and how often its responses are small. Each row also has a label y: 1 if the blocklist calls it a tracker, 0 if not.

from watched.leg7b_web import tracker_dataset
data = tracker_dataset(seed=2)
print(data["features"])
row = data["train"][0]
print(row["site"], row["x"], row["y"])

Logistic regression in three ideas

Weight: a number saying how much one feature counts as evidence. Positive means "more of this, more tracker-like".

1. Weigh the evidence. Multiply each feature by its weight, add them up, and add a bias (a starting lean): z = w1*x1 + w2*x2 + ... + bias.

2. Squash it into a probability. The sigmoid function 1 / (1 + math.exp(-z)) turns any number into one between 0 and 1. Big positive z gives nearly 1, big negative nearly 0, zero gives 0.5.

3. Learn from mistakes. Start with all weights 0. For each training row, compute error = prediction - y. Then nudge each weight: w = w - lr * error * x, and the bias by bias - lr * error. lr (the learning rate) sets the nudge size. Going through all rows once is an epoch; repeat for many epochs.

import math
def sigmoid(z):
    return 1 / (1 + math.exp(-z))
print(sigmoid(0), round(sigmoid(4), 3), round(sigmoid(-4), 3))   # 0.5 0.982 0.018

Judging it: precision and recall

You met these in Leg 2. Say "tracker" when the probability is at least 0.5. Then:

  • Precision = true trackers flagged / everything flagged. Low precision breaks harmless sites.
  • Recall = true trackers flagged / all true trackers. Low recall lets trackers through.

Always measure on held-out rows, ones the model never trained on. Scoring on the training rows flatters it. The tests go further and score your model on a crawl of a web it has never seen.

A common misconception: "The model learned what a tracker is." It learned to agree with the blocklist that labelled the data, mistakes included. If the list is wrong about a site, the model is taught to be wrong too. Lesson 149 comes back to this.

In other languages

The prediction and one learning step, elsewhere:

// JavaScript
const predict = (w, b, x) => 1 / (1 + Math.exp(-(x.reduce((s, v, i) => s + v * w[i], 0) + b)));
function step(w, b, row, lr) {
  const err = predict(w, b, row.x) - row.y;
  return [w.map((wi, i) => wi - lr * err * row.x[i]), b - lr * err];
}
// C#
double Predict(double[] w, double b, double[] x) {
    double z = b;
    for (int i = 0; i < x.Length; i++) z += w[i] * x[i];
    return 1.0 / (1.0 + Math.Exp(-z));
}

Recap

  • A classifier learns tracker behaviour, so it can flag trackers no list names yet.
  • Logistic regression: weigh the features, squash with the sigmoid, nudge weights by the error.
  • Judge it by precision and recall on rows it never trained on.
  • It learns the labels' view of the world, mistakes included.
Your turn

Write three functions, in plain Python:

  1. predict(weights, bias, x): the sigmoid 1 / (1 + math.exp(-z)) of z = sum(weights[i] * x[i]) + bias.
  2. train(rows, epochs=100, lr=0.5): start with zero weights (one per feature) and bias 0.0; for each epoch, for each row in order, compute error = predict(...) - row["y"], then update every weight by - lr * error * x[i] and the bias by - lr * error. Return (weights, bias).
  3. precision_recall(weights, bias, rows, threshold=0.5): flag a row when the prediction is at least the threshold; return (precision, recall), using 0.0 when a division would be by zero.
Then set weights, bias from data["train"] and test_precision, test_recall on data["test"]. Tests score your model on a simulated web it never saw: at least 0.80 for both.

Spotted a problem in this lesson? Report it

Code · runs in your browser
Output

        

      

      
      

        Back