Chapter 19 of 36 · ~1 min
Observability
When an agent fails, "the AI gave a bad answer" is not a diagnosis. Was the prompt wrong? Did retrieval return the wrong passage? Did the model call the wrong tool, or the right tool with wrong arguments? Did a tool return an error the model then paraphrased as a fact? Each has a different fix, and you can only tell which happened if you recorded the whole execution.
Trace everything: the prompt as sent, the context retrieved, the model's decision, each tool call and its result, each subsequent decision, and the final output. This is ordinary logging applied to an unusual component, and it is the difference between debugging and guessing.
Exercise
Do it yourselfInspect the full trace of a run that produced a wrong answer and locate the step where it went wrong.
Big question
If you could see only one thing from a failed run, what would it be?
