Confident and wrong
Why it matters A language model writes a wrong answer in the same calm, well-formatted voice as a right one. Tone tells you nothing. A test that fails tells you everything.
Fluency is what these models are trained for, so it costs them nothing. Correctness is a separate property, and the text does not show which one you are getting. Advice that sounds like settled practice ("just retry it", "this regex is the standard one", "name and size are enough") is the most dangerous, because it does not invite a second look.
An early-2025 study by METR found that experienced open-source developers using AI tools took longer on their tasks while believing they had been faster (METR 2025). How confident you feel about an answer is not a measurement of it.
Prove it, do not argue it
When advice feels off, the strongest reply is not a better argument. It is a small test that fails on the advised code and passes on a correct version. An argument can be argued back. A failing test ends the discussion, and it stays in the codebase so the mistake cannot return.
Counterexample: one concrete input where the claim is false. One is enough to sink "this always works".
How this module works
Each lesson shows a piece of planted advice with code. The advice was written for the course and is wrong on purpose, in ways that real advice is wrong. You write a test. It runs against the advised code, several close relatives of it, and a hidden correct version. It must pass the correct one and catch enough of the others.
Recap Fluent is not correct. Turn a doubt into a counterexample, and turn the counterexample into a test.
Spotted a problem in this lesson? Report it · Ask about this lesson
This exercise opens soon
Finish the lesson before it and this one unlocks. The whole course is free, no account needed, and you can read this lesson right now.
This browser is blocking cookies for learn.floati.life, so lessons you finished here cannot open in order. Allow cookies for this site and reload.