Phase 3 — Core AI Engineering · Lesson 36 · 20 XP
Observability and tracing
A RAG or agent pipeline has many steps — retrieval, one or more tool calls, one or more model calls. When the final answer is wrong, the final output alone doesn't tell you which step broke. Tracing captures each intermediate step's input, output, latency, and token count, so you can see exactly where things went sideways.
Dedicated tools like Langfuse and LangSmith build this in for LLM pipelines specifically; plain OpenTelemetry spans work too if you'd rather not add a specialized dependency. Either way, log at minimum: what was sent, what came back, how long it took, and how many tokens it used — for every model call.
Exercise
Instrument your Phase 3 RAG or agent pipeline with tracing that captures each step's input, output, latency, and token count. Use it to diagnose one deliberately slow or wrong run — find the actual step that caused the problem.
Check yourself
1. Why is tracing each intermediate step more useful for debugging an agent than only logging the final output?
2. What three pieces of information are worth capturing for every model call in a trace?
MCP (Model Context Protocol)
Answer the check-yourself questions to unlock this