Hashing is not anonymising
Why it matters Companies often say they share only "hashed" email addresses, as if that made them anonymous. In this lab you undo a hashed list in a fraction of a second, on synthetic emails. Once you have seen it, you will read "we only share hashed data" very differently.
A one-way blender
Think of a hash as a blender: put in a strawberry and you get the same pink smoothie every time, but you cannot turn the smoothie back into a strawberry. That sounds safe. The catch: if you suspect it was a strawberry, you blend one yourself and compare.
Hash: a function that turns any text into a fixed-length fingerprint, the same every time for the same input. SHA-256 gives 64 hex characters. You cannot run it backwards.
Brute force: trying every possible input until one matches. It works whenever the list of possible inputs is small enough to try.
Why it is not anonymous
Emails are hashed precisely so two companies can match them: both hash sam.smith@mail.example and get the same 64 characters. Hashed-email IDs in advertising work this way. The hash is a stable ID for the person, not a disguise.
And there are not that many emails. Our practice mail system only hands out first.last@domain or firstNN@domain, from 30 first names, 30 last names and 3 domains: 11,610 possible addresses in total. Hashing all of them takes a blink. Real address spaces are much bigger, but data brokers already hold huge lists of real addresses, so they only hash what they have and compare.
The plan
normalise(email): strip spaces and lower-case. Everyone does this before hashing, soSam.Smith@Mail.Exampleandsam.smith@mail.examplematch.sha256_hex(text):hashlib.sha256(text.encode("utf-8")).hexdigest()..encodeturns text into bytes, which is what hash functions eat.all_candidates(): loop over domains, first names and last names to list every possible address.crack(hashes): build a dict from each candidate's hash to the candidate, once, then look each leaked hash up. Building it once matters: hashing 11,610 addresses for every one of 200 leaked hashes would be 200 times slower.
Lookup table: a dict built in advance from answers you might need, so each question is answered in one step. Attackers build them for common passwords and email lists.
A common misconception
"SHA-256 is unbreakable, so hashed data is safe." SHA-256 is fine at what it does: nobody runs it backwards. The weakness is the input: when the possible inputs can be listed, you never need to run it backwards. Phone numbers are a well-known case: all the numbers in one country fit into a few billion possibilities, few enough for a fast computer to try every one.
Reminder: this only ever runs on synthetic addresses. Doing it to real people's data is exactly the harm this module teaches you to prevent, and the next lesson is the defence.
Try it Time it: wrap your crack call with time.perf_counter() before and after. On most laptops it finishes in well under a second for 11,610 candidates.
In other languages
JavaScript (Node):
const { createHash } = require("crypto");
const sha256 = t => createHash("sha256").update(t, "utf8").digest("hex");
const lookup = new Map(allCandidates().map(e => [sha256(e), e]));
const found = leaked.filter(h => lookup.has(h)).map(h => [h, lookup.get(h)]);PHP:
$lookup = [];
foreach (all_candidates() as $e) { $lookup[hash('sha256', $e)] = $e; }
$found = array_intersect_key($lookup, array_flip($leaked));Recap
- A hash is the same for the same input, which makes it a stable ID, not a disguise.
- When the possible inputs can be listed, hash them all and compare: brute force.
- Build the lookup table once, then each hash is a single dict look-up.
- "We only share hashed emails" still shares a person. The defence is next.
Write normalise(email) (strip and lower-case), sha256_hex(text) (SHA-256 hex digest of the UTF-8 text), all_candidates() (a list of every first.last@domain and firstNN@domain address, NN from 1 to 99, using FIRST_NAMES, LAST_NAMES and MAIL_DOMAINS) and crack(hashes) (a dict {hash: email} for each hash you can recover). Then set found = crack(leaked).
This lesson is locked
Lessons open one at a time. Finish the previous lesson to unlock this one.