Syllabus Lesson 57 of 156 · Measure privacy, honestly
Measure privacy, honestly

Leg 2 project: the Tracker Lab report

Why it matters This is where Leg 2 comes together. You run the full loop, tracker, then partition, then blocker, then measurement, on a practice world you have not seen before, and you report it the way a privacy researcher would: as numbers anyone can check.

The brief

A practice newsroom has asked you: "How much does our readers' browser choice protect them on these three shops?" You will answer with a JSON report produced by your own code. JSON is just a dict written as text, so another program (or a grader) can read it.

Lab world: build_lab_world(seed) from watched.leg2b_lab: the usual three shops, AdCo and Metrics, plus one extra tracker whose name changes with the seed, and a harmless reviews widget on every shop, there to tempt a careless blocker.

Because the extra tracker changes with the seed, a hard-coded list will not pass. Your detector from lesson 48 has to find it. lab_truth(world) gives you the answer key for scoring only: in real research, building it is slow handwork.

Three runs

Each run browses the same list of pages from lab_urls(world, n, seed), with a fresh copy of AdCo's practice tracker installed (server, profiles = practice_tracker(world) from watched.leg2e_inside, the same ready-made tracker you studied in Module 7), so no run sees another's profiles. You do not write the tracker; you measure it.

  1. Plain browser. How many shop sites does AdCo's tracker join to one ID? Also run find_trackers on this run's log.
  2. Partitioned browser (Browser(world, partitioned=True)). Same question. Each shop gets its own AdCo cookie jar, so the answer should drop to 1.
  3. Your blocker (Browser(world, blocker=make_blocker(found))). Same question, plus precision and recall against lab_truth(world).

A tidy way to avoid repeating yourself: write one helper, for example tracked_run(world, urls, **options), that removes old trackers, installs a fresh one, browses, and returns the browser and the linked count. **options passes keyword settings like partitioned=True straight on to Browser.

The report

KeyValue
pagesnumber of URLs browsed
trackers_foundsorted list of sites your detector flagged in run 1
linked_sites, linked_sites_partitioned, linked_sites_blockedthe most shop sites joined to one AdCo ID in runs 1, 2 and 3 (0 if no profile)
requests_totalnumber of records in run 3's log
requests_blockedhow many of them were blocked
precision, recallfor run 3 against lab_truth(world), rounded to 3 places

Finally set report_json = json.dumps(report, indent=2, sort_keys=True) for build_lab_world(seed=7) with lab_urls(world, 30, seed=7), and print it.

Checklist before you submit

  • Each run installs a fresh tracker and removes it afterwards (world.remove_trackers()).
  • find_trackers does not flag widgets.example or cdn.example.
  • Every value in the report is a plain number, string or list, so json.dumps works.

Reading your own results

You should see the story of the whole leg in a few numbers: one ID across every shop, then 1 with partitioning, then 0 with your blocker. Partitioning alone stops the cross-site join, but AdCo still sees each visit. Blocking stops AdCo hearing anything, at the risk of false positives, which your precision shows. Neither touches what a shop's own server shares. That gap is where Legs 3 and 4 pick up.

Try it When you pass, make a careless copy of your detector that flags every third-party site, ignoring the clues, and rerun the blocker with it. Watch precision fall: which harmless sites got caught, and what would break on a real shop?

In other languages

JavaScript:

// The same report, written out as JSON
const report = {
  pages: urls.length,
  trackers_found: [...found].sort(),
  linked_sites: linked,
  precision: Math.round(precision * 1000) / 1000,
};
console.log(JSON.stringify(report, null, 2));

Go:

type Report struct {
    Pages         int      `json:"pages"`
    TrackersFound []string `json:"trackers_found"`
    LinkedSites   int      `json:"linked_sites"`
    Precision     float64  `json:"precision"`
}
out, _ := json.MarshalIndent(report, "", "  ")
fmt.Println(string(out))

Recap

  • The full loop: track, partition, block, measure, on a world you had not seen.
  • Partitioning stops cross-site joins; blocking stops the tracker hearing at all.
  • Precision and recall keep your blocker honest about breakage and leaks.
  • A report of checkable numbers is how privacy claims should be made.
Your turn

Write find_trackers(log) (from lesson 48, with its helpers), score(log, trackers) (from lesson 51), make_blocker(trackers) returning a blocker that blocks every request whose site is in trackers, and run_lab(world, urls) returning a report dict with exactly the keys pages, trackers_found, linked_sites, linked_sites_partitioned, linked_sites_blocked, requests_total, requests_blocked, precision and recall, as described in the brief. Then set and print report_json for build_lab_world(seed=7) and lab_urls(world, 30, seed=7). AdCo's tracker is ready-made: use server, profiles = practice_tracker(world) from watched.leg2e_inside in each run.

Spotted a problem in this lesson? Report it

Code · runs in your browser
Output

        

      

      
      

        Back