Cut Off, Refused, or Out of Scope
Why it matters A support bot that shows a user half a sentence, or an empty bubble, looks broken. Before you show any reply, decide what kind of reply it is.
The app here is a bank assistant whose system prompt says: answer only questions about accounts and cards, and for anything else reply with exactly OUT_OF_SCOPE. Five replies:
- a normal answer;
OUT_OF_SCOPEexactly as asked;- the same signal written loosely (
Out_of_scope.with a newline): models follow "reply exactly" almost always, so match it with some tolerance (ignore letter case, spaces and a trailing full stop); - an answer cut off at the token limit;
- a refusal:
stop_reasonis"refusal", the content is empty, and astop_detailsdict may say why.
Each kind needs a different screen: show the answer, show "I can only help with banking", ask the model to continue or retry with a higher limit, or show a polite decline.
About these replies: they are simulated. Each one was written for this course in exactly the shape the Claude Messages API returns (content blocks, stop_reason, usage), and each copies something real models do. They are not recordings of a live model, so the exact wording is ours; the shapes and the problems are the real ones your code will meet.
Write classify_reply(reply) returning one of "refused" (stop_reason is "refusal"), "cut_off" (stop_reason is "max_tokens"), "out_of_scope" (the text, with surrounding spaces and a trailing . removed, equals OUT_OF_SCOPE ignoring case), "empty" (no text at all), or "answer". Check them in that order. Keep the starter's REPLIES as it is: the check reads it.
Spotted a problem in this lesson? Report it · Ask about this lesson
This exercise opens soon
Finish the lesson before it and this one unlocks. The whole course is free, no account needed, and you can read this lesson right now.
This browser is blocking cookies for learn.floati.life, so lessons you finished here cannot open in order. Allow cookies for this site and reload.