Syllabus Lesson 94 of 156 · Threat modelling
Threat modelling

Product threat models

Why it matters When you build an app, privacy problems hide in the arrows: the moments data moves from one place to another. Writing those moves down as data, then letting code check them, finds the risky ones before your users do. It is the habit privacy engineers use on real products.

Following the parcel

Think of each piece of personal data as a parcel. A threat model for a product asks: where does each parcel go, is it sealed on the way, who receives it, and how long do they keep it? A data flow is one parcel's journey.

Data flow: one movement of data from one place to another, for example "the sign-up email goes from the app to our server" or "page views go from the app to an analytics service".

Third party: any company or service that is not you or your user. An analytics provider, an ad network, an email-sending service.

Retention: how long data is kept before it is deleted. "Forever" is a retention period too, and usually the worst one.

Flows as Python data

Each flow fits naturally in a dictionary. Here is one from a made-up plant care app, plantly.example:

flow = {
    "id": "photo-upload",
    "data": "plant photos with location",
    "personal": True,          # does it point to a person?
    "to": "plantly.example",
    "third_party": False,      # is the receiver someone other than us?
    "encrypted": True,         # sent over HTTPS (or similar)?
    "retention_days": 90,      # None means kept forever
    "purpose": "identify plant disease",
}

One handy tool: flow.get("retention_days") reads a key like flow["retention_days"] does, but returns None instead of crashing when the key is missing. The tests use flows with missing keys, so you will need it.

A list of these dictionaries is a whole product's data map. Once it is data, you can check every flow with the same rules, every time anything changes.

Four warning signs

Only personal flows matter here: a flow that carries nothing about a person cannot leak anything about one. For a personal flow, flag it when:

  1. not encrypted: anyone on the network path can read it;
  2. third party: it leaves your control, and you depend on another company's habits;
  3. kept too long: retention is missing (forever) or more than 365 days;
  4. no purpose: nobody wrote down why it is collected. If you cannot say why, you probably should not collect it.

A flagged flow is not automatically wrong. Sending emails through an email service is a third-party flow you may well keep. The flag means "a human must look at this and decide", which is exactly what a threat model is for.

A common misconception

"If it is encrypted, it is safe." Encryption protects the trip. It says nothing about who receives the data or how long they keep it. A flow can be encrypted and still be the riskiest one in your app.

In other languages

JavaScript: the same rules over an array of objects.

function riskReasons(f) {
  if (!f.personal) return [];
  const r = [];
  if (!f.encrypted) r.push("not encrypted");
  if (f.third_party) r.push("third party");
  if (f.retention_days == null || f.retention_days > 365) r.push("kept too long");
  if (!f.purpose) r.push("no purpose");
  return r;
}
const flagged = Object.fromEntries(
  flows.map(f => [f.id, riskReasons(f)]).filter(([, r]) => r.length)
);

Go: a struct with a pointer for "maybe missing" retention.

type Flow struct {
    ID, Purpose                     string
    Personal, ThirdParty, Encrypted bool
    RetentionDays                   *int // nil means forever
}

Recap

  • A data flow is one movement of data; a list of flows is a product's data map.
  • Writing flows as dictionaries lets code check every one with the same rules.
  • Flag personal flows that are unencrypted, go to a third party, are kept too long or have no purpose.
  • A flag means "a human decides", and encryption alone does not make a flow safe.
Your turn

The starter code holds FLOWS, the data map of the practice app plantly.example. Write two functions:

  1. risk_reasons(flow) returns a list of warning strings for one flow. If flow["personal"] is false, return an empty list. Otherwise add, in this order:
    • "not encrypted" if flow["encrypted"] is false;
    • "third party" if flow["third_party"] is true;
    • "kept too long" if retention_days is missing, None, or more than 365;
    • "no purpose" if purpose is missing or an empty string.
  2. flag_risky(flows) returns a dictionary mapping each flagged flow's "id" to its list of reasons. Flows with no reasons are left out.
Use flow.get(...) so a missing key does not crash. Finally, print each flagged id and its reasons, one per line.

Spotted a problem in this lesson? Report it

Code · runs in your browser
Output

        

      

      
      

        Back