Syllabus Lesson 150 of 156 · Measure the web like a researcher
Measure the web like a researcher

Leg 7 project: a study with a fix

Why it matters Real privacy work is a loop: measure a problem, apply a fix, measure again, and say honestly what changed and what it cost. This project runs that loop end to end, joining the crawler from this module with a defence from the last one.

Think of a doctor checking whether a treatment works: measure first, treat, measure again the same way, and note the side effects. Without the first measurement there is nothing to compare against; without the side effects the result is only half true.

The question

On the simulated web, could someone watching your encrypted connection tell which sensitive site you visited, and how much padding would it take to stop them reliably?

Visiting a health, finance or religion site can reveal something personal even if nothing else leaks. You will:

  1. Measure the setting. Crawl SimWeb(seed), find the sites whose category is in SENSITIVE, and record what share of them load a listed tracker. Take the first 6 sensitive sites by rank.
  2. Measure the problem. Get page loads for those 6 sites with page_loads(seed, sites=...) and measure the nearest-centroid accuracy, as in lesson 143.
  3. Apply the fix. Try padding budgets in PAD_LEVELS in order. For each, pad the training loads with seed=1 and the test loads with seed=2, retrain, and measure again. Stop at the first budget whose accuracy is at or below TARGET. If none reaches it, keep the last one.
  4. Measure the cost. The overhead at the chosen budget, as in lesson 143.
  5. Write limits. At least two honest sentences, as in lesson 149.

The report

study(seed) returns a dict with exactly these keys:

{
  "seed": 7,
  "sites": ["...", ...],              # the 6 sensitive sites, by rank
  "sensitive_with_trackers": 0.97,    # share of ALL sensitive sites with a listed tracker
  "accuracy_before": 0.94,
  "padding_budget": 800,              # the first level in PAD_LEVELS reaching TARGET
  "accuracy_after": 0.19,
  "overhead": 2.07,                   # extra test cells as a fraction: here about 3 times the traffic
  "target_met": True,                 # accuracy_after <= TARGET
  "limits": ["...", "..."]            # at least two sentences
}

The numbers are graded against a hidden run on another seed, so your code has to do the study, not copy one answer. The limits are not graded for content, but the tutor can give feedback on them. Think about: a closed world of 6 sites, an observer who knows your padding settings, simulated traffic, and what the tracker share does and does not tell you.

Notice the two halves of the result. Padding protects the visit from a network observer, but the trackers on those same sites still learn about you directly. One defence never covers every threat, and your write-up can say so.

A common misconception: "If accuracy fell below the target, the problem is solved." You solved it for this observer, in this closed world, at a measured cost. That is a real result, and it has edges.

In other languages

Choosing the first level that meets a target, elsewhere:

// JavaScript
let chosen, after;
for (const budget of PAD_LEVELS) {
  chosen = budget;
  after = paddedAccuracy(data, budget);
  if (after <= TARGET) break;
}
// Go
var chosen int
var after float64
for _, budget := range padLevels {
    chosen, after = budget, paddedAccuracy(data, budget)
    if after <= target { break }
}

Recap

  • Measure, fix, measure again the same way, and report the cost.
  • Padding hid which sensitive site was visited from a network observer, at a measured bandwidth price.
  • The trackers on those sites are a separate problem that padding does not touch.
  • Honest limits are part of the result, not an apology for it.
Your turn

Write study(seed) following the five steps above, returning the report dict with exactly the keys shown. Use PAD_LEVELS = [50, 100, 200, 400, 800] and TARGET = 0.4, nearest-centroid classification on (outgoing, incoming) features with ties going to the alphabetically first site, pad_traces(..., seed=1) for training loads and seed=2 for test loads. Set report = study(7). The tests rerun your study on a hidden seed.

Spotted a problem in this lesson? Report it

Code · runs in your browser
Output