Syllabus Lesson 125 of 156 · Build the defences
Build the defences

Breakage

Why it matters Every defence you built in this module breaks something real when it is set too hard. People do not switch off protection because they love trackers; they switch it off because their comments box or checkout stopped working. So an engine team measures two numbers together: how much tracking still works, and how many sites broke. This lab gives you both numbers and asks you to choose.

It is like choosing a medicine dose. Too little and it does nothing; too much and the side effects are worse than the illness. The right dose is the smallest one that works, and you only find it by measuring.

Breakage: a site feature that stops working because of a privacy defence: a sign-in that fails, a widget that shows logged out, a chart that draws wrong.

Breakage rate: the share of tested sites with at least one broken feature. Three broken sites out of 20 is a rate of 3 / 20 = 0.15, or 15 percent.

Breakage budget: the highest breakage rate a team will accept before shipping, for example 0.10 (two sites in 20).

The test bench: 20 practice sites

watched.leg6b_harden builds 20 practice sites (news.example, garden.example and so on). Each needs some of these features to work:

FeatureWhat it needsWhich defence can break it
liba script from cdn.exampleblocking every third party
mapsa map frame from maps.exampleblocking every third party
commentsthe Talk widget, signed in after a clickpartitioning without Storage Access
chartcanvas pixels within 2 of the truthfarbling with too much noise
checkoutpay.example receives its reflink stripping without an allowlist
loginsignin.example receives cid, state and codelink stripping without an allowlist
linkslinks between sites landblocking navigations

Five tracking techniques run across the same sites: a third-party cookie, frame storage, a fingerprint script, link decoration and a bounce tracker. A technique counts as linked when it saw the same ID on two or more sites.

Ready-made defences

You wrote each defence yourself earlier in this module. To compare combinations quickly, watched.leg6b_presets has them ready-made, switched by a config dict, and CANDIDATES holds eight named configs, from "nothing" to "balanced". One call measures a config:

from watched.leg6b_presets import CANDIDATES, evaluate

result = evaluate(CANDIDATES["blocklist only"])
print(result["linked"])   # techniques that still link two sites
print(result["broken"])   # {site: [features that broke]}
print(result["sites"])    # 20

What you will build

  • breakage_rate(broken, total_sites): the share of sites with anything broken.
  • feature_counts(broken): for each feature, on how many sites it broke. This tells you which defence to soften.
  • choose(results, budget): from a dict of name -> result, keep the configs whose breakage rate is within the budget, then pick the one with the fewest linked techniques. On a tie, prefer the lower breakage rate, then the name that comes first alphabetically. Return None if nothing fits.

To pick "the fewest, then the lowest, then the first name" in one step, sort by a tuple. Python compares tuples item by item, so the first item decides unless it is a tie:

options = [("b", 0, 0.20), ("a", 0, 0.20), ("c", 1, 0.00)]
best = min(options, key=lambda o: (o[1], o[2], o[0]))
print(best)   # ('a', 0, 0.2): fewest linked first, then rate, then name

key=lambda o: ... tells min what to compare. A lambda is a one-line function with no name: lambda o: o[1] means "given o, give back o[1]".

Reading your results

Look at the table you print. Some configs stop all five techniques but break a fifth of the web; one stops four and breaks nothing. Neither is wrong in itself: the budget decides. Real engines work the same way. They roll defences out gradually, collect breakage reports, keep small lists of exceptions for sites that need them, and let people turn protection down for one site. Floati's per-site Shields are that last switch.

A common misconception

"Zero breakage means the defence is safe to ship." Not on its own: the config "nothing" has zero breakage too. Breakage is only meaningful next to how much tracking you stopped, which is why choose needs both.

In other languages

Choosing within a budget in JavaScript:

function choose(results, budget) {
  const ok = Object.entries(results)
    .map(([name, r]) => [name, r.linked.length, Object.keys(r.broken).length / r.sites])
    .filter(([, , rate]) => rate <= budget)
    .sort((a, b) => a[1] - b[1] || a[2] - b[2] || a[0].localeCompare(b[0]));
  return ok.length ? ok[0][0] : null;
}

Counting features in Go, with a map:

counts := map[string]int{}
for _, features := range broken {
    for _, f := range features { counts[f]++ }
}

Recap

  • Every defence has a breakage cost; measure it, do not guess.
  • Judge a config on two numbers together: tracking still working and breakage rate.
  • Count which feature breaks most, and soften the defence behind it rather than dropping it.
  • A breakage budget turns a vague trade-off into a decision you can test.
Your turn

Write breakage_rate(broken, total_sites) (number of sites in broken divided by total_sites), feature_counts(broken) (a dict: feature -> number of sites where it broke), and choose(results, budget), where results maps a config name to what evaluate returned. choose keeps names whose breakage_rate(r["broken"], r["sites"]) is at most budget and returns the one with the fewest r["linked"]; ties go to the lower rate, then the alphabetically first name; return None if none fits. Then build results = {name: evaluate(config) for name, config in CANDIDATES.items()} and set best = choose(results, 0.10).

Spotted a problem in this lesson? Report it

Code · runs in your browser
Output