Syllabus Lesson 95 of 141 · The lying lesson
The lying lesson

Lie 3: same name, same size, same file

Why it matters A duplicate finder is usually followed by a delete. If it calls two different files "the same", somebody's work is gone, and nothing in the output warned them.

Planted advice: the advice below was written for this lesson, and it is wrong on purpose. It is the kind of answer an AI assistant gives fluently and confidently. Your job is not to argue with it but to prove the flaw with a test.

import os

# Hashing every file is slow. Two files with the same name and the same size
# are the same file in practice, so group on that.
def find_duplicates(files):
    groups = {}
    for path, data in files:
        groups.setdefault((os.path.basename(path), len(data)), []).append(path)
    return sorted(sorted(g) for g in groups.values() if len(g) > 1)

The speed argument is real: reading every byte of every file is the slow part. But it is answering a different question. Same name and size is a cheap way to find files that might be duplicates. Only the contents can say they are.

Think about what happens in real folders: report.docx saved in two project folders with different text and, by chance, the same size; a photo copied and renamed IMG_0001 (copy).jpg; a config file where only one byte differs.

Candidate filter: a cheap check that rules things out but cannot rule them in. Size is a good one: files of different sizes cannot be equal. Equal sizes only mean "worth comparing".

Your turn

Write my_test(find_duplicates). It receives a version of the function and must raise if it breaks a rule. Files are given as a list of (path, data) pairs, where data is the file's bytes. The rules: two files are duplicates when their bytes are identical, whatever their names; return a list of groups, each a sorted list of the paths of 2 or more identical files, and the groups sorted; files with no duplicate do not appear. Pass bar: pass the correct version and catch at least 4 of the 6 broken versions (the planted advice is one of them).

Spotted a problem in this lesson? Report it · Ask about this lesson

This exercise opens soon

Finish the lesson before it and this one unlocks. The whole course is free, no account needed, and you can read this lesson right now.

Start at lesson 1 Back to syllabus