Syllabus Lesson 56 of 141 · Speed without the cache
Speed without the cache

Stream it

Why it matters Logs, metrics and exports are often too big to hold. The default "load it, then work on the list" fails the day the file outgrows the machine. A running summary never does.

A gateway writes one record per request: (request_id, status, ms). Every hour a small job reads the hour's log, which arrives as a stream you can read once, and posts a summary to the team's channel: how many requests, how many errors, the average and longest duration, and the five slowest requests so someone can look at them.

Stream: data you see one item at a time, in order, and cannot rewind.

Your turn

Write summarise(tape). tape is a Tape of (request_id, status, ms) records (all whole numbers). Return a dict with: count; errors (status 500 or above); mean_ms; max_ms (both 0 for an empty log); and slowest, a list of up to five (ms, request_id) pairs, longest first, equal durations with the smaller id first. Budgets: read the tape once (30,000 reads for 30,000 requests) and use at most 16,000 bytes of working memory while your function runs (measured with tracemalloc; holding or sorting all requests is far over).

Spotted a problem in this lesson? Report it · Ask about this lesson

This exercise opens soon

Finish the lesson before it and this one unlocks. The whole course is free, no account needed, and you can read this lesson right now.