Clean links at navigation

Picture a post office that peels the advertising stickers off every envelope before delivery. It is a good service, until it peels off the order number a shop needs to find your parcel. So the post office keeps a short list: "stickers named ref stay on envelopes for the payment office".

Why at navigation time, and only there

In Leg 2 you wrote a link cleaner as a function. Inside an engine it runs on every navigation, including each redirect hop. That matters: a click server such as deco.example/out?to=... can add adco_id to the link on its way through, after you clicked. Only a cleaner that sees every hop removes it.

Engines leave the addresses of pictures, scripts and frames alone. A page's own scripts can send an ID however they like, so cleaning those addresses protects little and breaks a lot.

Some real browsers do this today (checked October 2026). For example, Firefox strips a list of known tracking parameters in Strict mode and in private windows, Safari strips some in Private Browsing, and Brave strips a list of known parameters by default. Each list is short and chosen with care, for the reason the next section explains.

Why the allowlist exists

Some names are only ever trackers, like fbclid or utm_campaign. Short, general names are trickier. In the practice world, ref is an affiliate tag on ShopC but the order reference on pay.example; cid is an analytics ID on ShopA but the app's sign-in ID on signin.example. Strip them everywhere and payments and sign-ins fail. The module gives you:

TRACKING_PARAMS = {"fbclid", "gclid", "msclkid", "mc_eid", "adco_id", "ref", "cid"}
TRACKING_PREFIXES = ("utm_",)            # every name starting with utm_
ALLOWLIST = {"pay.example": {"ref"}, "signin.example": {"cid"}}

Compare names in lower case: UTM_Source is still a campaign tag. name.lower() gives the lower-case text, and name.startswith(TRACKING_PREFIXES) checks every prefix in the tuple at once.

Taking a URL apart and putting it back

Python's urllib.parse does the fiddly parts. Never cut URLs up with plain text tricks: values can contain encoded characters such as %20.

from urllib.parse import urlsplit, parse_qsl, urlencode, urlunsplit

parts = urlsplit("https://shopa.example/p/tent?q=tent&utm_source=news#reviews")
print(parts.scheme, parts.hostname, parts.path)   # https shopa.example /p/tent
pairs = parse_qsl(parts.query, keep_blank_values=True)
print(pairs)                                      # [('q', 'tent'), ('utm_source', 'news')]
kept = [(k, v) for k, v in pairs if k != "utm_source"]
print(urlunsplit((parts.scheme, parts.netloc, parts.path, urlencode(kept), parts.fragment)))
# https://shopa.example/p/tent?q=tent#reviews

[(k, v) for k, v in pairs if ...] is a list comprehension: a short way to write "make a new list of the pairs that pass this test". When no pairs are left, urlencode([]) is empty and urlunsplit drops the ? for you. parts.netloc is the host plus any port; the site for the allowlist comes from site_of(url).

Plugging it into the engine

The practice browser takes a blocker(request) that sees every request, redirect hops included. It may return "allow", "block", or a new URL to use instead. request.kind is "document" or "form" for navigations. Your blocker cleans those and allows everything else untouched. Notice it never returns "block" for a navigation: the person chose to go there, so the engine cleans the trip rather than cancelling it.

run_journeys(blocker) follows ten decorated links, through redirects, payments and sign-ins, and reports for each: status, leaked (tracking names the landing server still received) and missing (names the site needed but lost).

const TRACKING = new Set(["fbclid", "gclid", "msclkid", "mc_eid", "adco_id", "ref", "cid"]);
const ALLOW = { "pay.example": new Set(["ref"]), "signin.example": new Set(["cid"]) };
function cleanUrl(text, site) {
  const url = new URL(text);
  const keep = ALLOW[site] ?? new Set();
  for (const name of [...url.searchParams.keys()]) {
    const low = name.toLowerCase();
    if ((TRACKING.has(low) || low.startsWith("utm_")) && !keep.has(low)) url.searchParams.delete(name);
  }
  return url.toString();
}

// needs: using System.Linq; using System.Web;
// IsTracking(name, site) is your is_tracking, written in C#
var uri = new UriBuilder(url);
var query = HttpUtility.ParseQueryString(uri.Query);
foreach (var name in query.AllKeys.ToList())
    if (IsTracking(name, site)) query.Remove(name);
uri.Query = query.ToString();