Syllabus Lesson 46 of 156 · Build a blocker
Build a blocker

How blocklists work

Why it matters Most tracker blocking in the world today, in extensions and in browsers, starts with a plain text file of rules. Once you can read that file, you can see exactly what a blocker will and will not stop, and why a site sometimes breaks.

A bouncer with a list

Picture a bouncer at a club door holding a printed list: "do not let these people in", plus a short note: "except these regulars". The bouncer does not judge anyone; they just check names against the list.

A blocker works the same way. Every time a page asks for something (a script, an image, a pixel), the blocker checks the address against its list and says allow or block.

Filter list: a text file of rules, one per line, that tells a blocker which requests to stop. The best known family is EasyList, written in the Adblock Plus syntax that most blockers understand.

Request: one ask from your browser to a server, such as "send me https://adco.example/tag.js". One page usually makes many requests.

The syntax, a small practice subset

Real lists have tens of thousands of lines and many extra features. We use a small, made-up practice list with the same core syntax. Here it is (it lives in watched.leg2b_blocker.PRACTICE_LIST):

[Adblock Plus 2.0]
! Title: Watched Practice List (made up for the course)

||adco.example^
||metrics.example^$third-party
||trackmore.example^
/p.gif?
*/collect?*cid=
/beacon/*/hit$third-party

! Exceptions: things that look like trackers but are needed
@@||cdn.example^
@@/assets/p.gif?

! Cosmetic rules (hide page parts). A network blocker ignores these.
##.ad-banner
shopc.example##.sponsored
Line looks likeMeaning
! ... or [...]A comment or a header. Skip it.
||adco.example^A domain rule: block adco.example and every subdomain, such as img.adco.example. The ^ marks the end of the host name.
/p.gif?A pattern rule: block any address that contains this text. A * inside means "any characters here".
@@...An exception: allow it, even if a block rule also matches.
...$third-partyAn option: only apply this rule when the request goes to a different site from the page you are on.
##.ad-bannerA cosmetic rule: hide part of the page. It never stops a request, so a network blocker skips it.

Third party: a request to a different site from the one in your address bar. On shopa.example, a request to adco.example is third party; one to shopa.example itself is first party.

Turning a line into data

Programs do not want to re-read the text for every request. So the first job is parsing: turning each line into a small dict once, at start-up.

Parsing: reading text and turning it into structured data a program can use directly, like turning a written shopping list into ticked boxes on a form.

Our rule dict has four keys:

parse_rule("||metrics.example^$third-party")
# {'exception': False, 'kind': 'domain',
#  'value': 'metrics.example', 'third_party': True}

parse_rule("@@/assets/p.gif?")
# {'exception': True, 'kind': 'pattern',
#  'value': '/assets/p.gif?', 'third_party': False}

parse_rule("! just a comment")
# None

Two Python string tools do most of the work. line.startswith("@@") checks how a line begins. line.rsplit("$", 1) splits once at the last $, giving the rule text and the options, so "a$b".rsplit("$", 1) is ["a", "b"].

A common misconception

"Blocklists block ads." Many do, but the lists that matter for privacy (such as EasyPrivacy) target trackers: pixels, beacons and analytics scripts, which you never see on screen. A page can show no ads at all and still send your visit to five companies.

Try it In a desktop browser with a blocker extension installed, open the extension's settings and find its filter lists. Most let you view a list as plain text. Look for lines starting with || and @@: they are exactly the rules you are about to parse.

In other languages

JavaScript:

function parseRule(line) {
  line = line.trim();
  if (!line || line.startsWith("!") || line.startsWith("[")) return null;
  if (line.includes("##") || line.includes("#@#")) return null;
  const exception = line.startsWith("@@");
  if (exception) line = line.slice(2);
  let thirdParty = false;
  const at = line.lastIndexOf("$");
  if (at >= 0) {
    thirdParty = line.slice(at + 1).split(",").includes("third-party");
    line = line.slice(0, at);
  }
  const isDomain = line.startsWith("||");
  const value = isDomain ? line.slice(2).replace(/\^$/, "") : line;
  return { exception, kind: isDomain ? "domain" : "pattern",
           value: value.toLowerCase(), thirdParty };
}

Rust:

struct Rule { exception: bool, domain: bool, value: String, third_party: bool }

fn parse_rule(line: &str) -> Option<Rule> {
    let mut l = line.trim();
    if l.is_empty() || l.starts_with('!') || l.starts_with('[') || l.contains("##") {
        return None;
    }
    let exception = l.starts_with("@@");
    if exception { l = &l[2..]; }
    let mut third_party = false;
    if let Some((rule, opts)) = l.rsplit_once('$') {
        third_party = opts.split(',').any(|o| o == "third-party");
        l = rule;
    }
    let domain = l.starts_with("||");
    let v = if domain { l[2..].trim_end_matches('^') } else { l };
    Some(Rule { exception, domain, value: v.to_lowercase(), third_party })
}

Recap

  • A filter list is plain text: domain rules (||host^), pattern rules, exceptions (@@) and options ($third-party).
  • Cosmetic rules (##) hide page parts and never stop a request.
  • Parse once into simple dicts so matching later is fast and clear.
Your turn

Write parse_rule(line) that returns None for blank, comment (!), header ([) and cosmetic (## or #@#) lines, and otherwise a dict with keys exception, kind ("domain" or "pattern"), value (lower case, without ||, the trailing ^, @@ or the $options) and third_party. Then write parse_list(text) that returns the list of rules for every line, in order, leaving out the None results. Do not import the course's own parser.

Spotted a problem in this lesson? Report it

Code · runs in your browser
Output

        

      

      
      

        Back