How privacy studies work
Why it matters Most of what we know about web tracking, such as how many sites carry trackers or which companies reach the most people, comes from studies that load thousands of sites automatically and record what happens. Knowing how those studies work lets you judge a headline number, and run your own in the next lessons.
Think of a bird survey. Volunteers walk the same routes at the same time of year and count what they see. The count is only as good as the method: walk only in sunny parks and you will miss the owls. Web studies are the same. A number like "80% of sites carry a tracker" means nothing until you know which sites, from where, and how they were visited.
Crawler: a program that visits web pages automatically, one after another, and records what it finds.
Census study: a measurement that tries to visit every site in a large, defined set (for example the most popular million sites) rather than a handful, so it can say how common something is.
Instrumentation: extra recording added to a browser so a study can see every request, cookie and script call a page makes, not just what appears on the screen.
How a modern study works
- Choose the sites. Usually a public ranking of popular sites, sometimes a random sample, sometimes one category (news, health).
- Drive a real browser. Simple downloads miss most trackers, because trackers are loaded by scripts. Studies drive a real browser that runs the page's code.
- Record everything. Requests and responses, cookies set, which browser features each script used (for example reading the canvas, a fingerprinting clue), and redirects.
- Label and count. Match the third parties against tracker lists, then count how often each appears.
A well-known example is OpenWPM, an open-source measurement tool, used in a 2016 study that crawled the most popular million sites and found fingerprinting scripts and tracking at a scale nobody had shown before. Many later studies follow the same pattern.
Why the method changes the answer
- Home page only, or deeper? Inner pages (articles, checkouts) often load different trackers.
- Stateless or stateful? A stateless crawl starts each site with a fresh browser; a stateful one keeps cookies across sites, like a person does, and so can see cross-site tracking.
- Consent banners. A crawler that never clicks measures what happens before consent; one that clicks "accept" measures something else. Say which you did.
- Where the crawler sits. Sites show different things in different countries and to different devices.
- Bot detection. Some sites treat crawlers differently from people, so a crawl may undercount.
- Timing. The web changes weekly. A study is a snapshot with a date.
A common misconception: "A bigger crawl is always a better study." Size reduces random error but not bias. A million home pages visited from one data centre with no clicks can be precisely wrong about what real people experience.
In this module you crawl a simulated web of 1,000 made-up sites. It behaves like the real thing in the ways that matter for learning, and nothing real is contacted.
Try it Open your browser's developer tools (F12, or Cmd+Option+I on a Mac), go to the Network tab, and load a news site. Count the different domains in the list. You have just done a crawl of one site, by hand. Then reload with the Network tab's "Disable cache" box ticked and see whether the list changes.
Recap
- Privacy studies crawl many sites with an instrumented real browser and count what they record.
- Census studies aim to cover a whole defined set, such as the top million sites.
- Choices like home page only, consent clicks, location and date change the results, so they must be reported.
- Bigger is not automatically better: size cuts random error, not bias.
This lesson is locked
Lessons open one at a time. Finish the previous lesson to unlock this one.