How the hidden world grades you

Think of a driving test. You practise on the streets near your home until you know every corner. On the day, the examiner picks a route you have never driven. The rules of the road are the same, but the streets are new, so the only way to pass is to actually be able to drive.

Your capstone works the same way. You build and practise on the practice world you used all course. When you submit, your work is run against a hidden world: same rules, different sites, names, trackers and visitors. If you did the Leg 2 project, you have already met this idea there.

Two kinds of check

• A report graded against the hidden world (projects A, B, D and F). Your code looks at the hidden world and writes a JSON report. The grader compares your report with what it knows is really there.
• Hidden tests (projects C and E). The grader calls your functions with inputs you have not seen and checks the results, just like the exercises earlier in the course.

Project D also has a written part. The write-up gets feedback from the tutor, but it is not graded; the JSON result is. That keeps the pass mark about things a machine can check fairly.

What a report check looks like

Each project page gives the exact fields to use. The shape is always similar. Here is an illustrative report for project A, the audit:

{
  "site": "shopa.example",
  "third_parties": ["adco.example", "cdn.example"],
  "trackers": ["adco.example"],
  "cookies": [{"name": "uid", "domain": "adco.example", "third_party": true}],
  "csp_present": false
}

The grader asks a few plain questions about it. Is it valid JSON? Are the required fields there, with the right types? And for each finding: is it really in the hidden world?

Hits, misses and false alarms

A smoke alarm can be right or wrong in two different ways. It can stay quiet during a real fire (a miss), or it can go off when you only made toast (a false alarm). A good alarm keeps both low. Your findings are judged the same way.

Say your blocker, on the hidden world, sees 20 tracker requests and 80 ordinary ones. It blocks 19 of the trackers and also, by mistake, 1 ordinary request that a page needed. That is 19 hits, 1 miss and 1 false alarm. In Leg 2 you turned counts like these into precision (how many of the things you blocked were really trackers) and recall (how many of the real trackers you caught).

A pass rule for a project might read like the one from the course plan: "blocked at least 95% of tracker requests, no false positives". Here 19 of 20 is exactly 95%, but the one wrongly blocked request still fails it. In privacy tools both mistakes matter: a miss lets tracking through, and a false alarm breaks the page and teaches people to switch the tool off.

Same code, same score

Grading is deterministic: run the same code on the same hidden world and you always get the same result. Where a project checks speed, it counts operations against a budget, never the time on a clock, so a slow laptop is not marked down.

Check your own report before submitting

Most failed submissions are not wrong ideas; they are a missing field or a typo. A tiny self-check catches those:

import json

report = {"site": "shopa.example", "trackers": ["adco.example"], "csp_present": False}

def missing_fields(report, required):
    # return the required field names that the report does not have
    return [name for name in required if name not in report]

print(missing_fields(report, ["site", "third_parties", "trackers", "csp_present"]))
# ['third_parties']  -> add it before you submit
print(json.dumps(report, indent=2, sort_keys=True))

const report = { site: "shopa.example", trackers: ["adco.example"], csp_present: false };
const required = ["site", "third_parties", "trackers", "csp_present"];
console.log(required.filter(name => !(name in report)));  // [ 'third_parties' ]
console.log(JSON.stringify(report, null, 2));

type Report struct {
    Site       string   `json:"site"`
    Trackers   []string `json:"trackers"`
    CSPPresent bool     `json:"csp_present"`
}
out, _ := json.MarshalIndent(Report{Site: "shopa.example", Trackers: []string{"adco.example"}}, "", "  ")
fmt.Println(string(out))

var report = new { site = "shopa.example", trackers = new[] { "adco.example" }, csp_present = false };
Console.WriteLine(System.Text.Json.JsonSerializer.Serialize(report));

A common misconception

"My practice world score is 100%, so the hidden score will be too." Not always. If your code quietly depends on details of the practice world, such as a list of tracker names you copied from it, it will miss trackers in the hidden world. That is called overfitting: fitting the examples you saw instead of the rule behind them. Test your logic on a few practice worlds built from different seeds (for example build_world(seed=2) as well as the usual one) before you submit.