There is one question an interviewer will ask about any production system you describe, and it separates people who have operated something from people who have only built one: "how do you know it's broken?" Observability is the answer. It is the instrumentation that turns a system you hope is working into one you can see is working, and can debug when it isn't. This page teaches it the way the field does: by working one debugging incident end to end, because the instrumentation only makes sense once you have felt the failure it exists for.
An incident: the wrong answer from yesterday
Take the research-library assistant from the retrieval walkthrough, live at the customer for a few months. (The incident that follows is illustrative: the request, the documents, the timings, and the scores are invented; the shape is what to study.) One morning an advisor writes in:
"Yesterday the assistant told me our view on European industrials is neutral. We upgraded to overweight in May. What else is it getting wrong?"
This is the classic complaint, and on its face it cannot be debugged. You were not there, the answer is gone, and that last question, what else is it getting wrong, is the sound of trust starting to drain. The deterministic-system instinct, reproduce it and then fix it, fails here: you can ask the assistant the same question now, but a probabilistic system does not owe you the same output. If today's answer is right, that proves nothing about yesterday's; if it is wrong, you still cannot see why. So you debug the recorded instance instead of a reproduction you may never get. The only version of yesterday's answer that can be debugged is the one the system wrote down at the time. That record is a trace.
Pulling the trace
You look up yesterday's request by user and time and pull its trace: the full path of that single request through every stage, with what each stage received, what it produced, and how long it took.
trace_id: 9f3c-0212 2026-06-09 14:12 UTC total: 2,279 ms
[input] question received (t=0)
"What's our current outlook on European industrials?"
[retrieval] top 3 chunks by vector similarity (87 ms)
index build: 2026-01-30
[1] 0.81 eu-industrials-2026-01.pdf, p.2
"We remain neutral on European industrials;
order books are soft and pricing power limited…"
[2] 0.77 global-sectors-2026-01.pdf, p.9
"…industrials positioning unchanged this quarter…"
[3] 0.72 eu-macro-2026-01.pdf, p.4
"Euro-area manufacturing surveys remain weak…"
[guardrail] input pre-check (injection scan, topic scope) (12 ms)
result: pass
[model] model: frontier-large-2026-05, prompt: v14 (1,940 ms)
tokens: 3,180 in / 210 out
"The house view on European industrials is neutral,
reflecting soft order books and limited pricing
power [1], unchanged positioning [2], and weak
manufacturing surveys [3]."
[groundedness] answer supported by retrieved chunks? (240 ms)
score: 0.96 — pass
[response] returned to advisor
stage sum: 87 + 12 + 1,940 + 240 = 2,279 msThe diagnosis, read from the trace
Read the stages in order and the failure localizes itself. The guardrail pre-check passed, correctly: the question was in scope. The model behaved: its answer is a faithful, cited summary of exactly the three passages it was handed. The groundedness check passed, also correctly, and that verdict is worth pausing on: groundedness verifies the answer against what was retrieved, so it can never catch retrieval itself being wrong. Every stage downstream of retrieval did its job, on bad input.
The failure is two lines in the retrieval stage. The top chunk is the January edition of the industrials report, the one the advisor says was superseded in May, and the line above it says why: the index was last built at the end of January. The May report exists in the customer's document store; it has simply never been embedded and indexed. Retrieval cannot surface a document the index has never seen.
Without the trace, this incident closes as "the model hallucinated," a verdict that can be neither fixed nor disproved, and that costs you the advisor's trust. With the trace, it is a stale-index bug with a one-line remedy: re-run the ingestion job so the index reflects the current corpus. Two follow-ups mark an operator. First, schedule the rebuild, because an index refreshed by hand will go stale again. Second, add the advisor's question, paired with the passage from the May report that should answer it, to the eval suite, so the next stale index surfaces as a score drop in the daily run rather than in an advisor's inbox. The incident belongs to the silent-regression family the deployment page is built around, where behavior changed with no code diff, and the versioned index and scheduled eval taught there are what would have caught it weeks earlier. The trace is how you find such a failure after the fact. The deployment disciplines are how you stop meeting it this way.
What the incident generalizes to
The debugging session used one stream of instrumentation and exposed the need for two more. Production observability for an AI system is three streams, and the incident shows what each is for.
Traces — the unit of debugging
A trace records the whole path of a single request: the input, what retrieval returned and with what scores, which guardrails fired, the exact model and prompt version, tokens in and out, the timing of every stage, and the final response. It is the unit of debugging for a probabilistic system because the same input can produce different outputs, so the recorded instance is the only debuggable object there is. The plumbing here is standard engineering: the distributed-tracing conventions of ordinary services apply directly. OpenTelemetry is the vendor-neutral standard, and the LLM-specific tracing products build on the same idea, adding the retrieval, prompt, and token fields the incident relied on.
Quality signals — notice before the user does
The worst fact in the incident is the channel it arrived by: a user complaint, weeks after the answers went stale. Nothing operational moved in those weeks. Latency was flat, error rate flat, uptime clean, because none of those metrics reads the content of an answer. Noticing quality decay takes signals built for it: the daily score from the online evals run against a fixed question set, and the behavioral proxies of live traffic such as thumbs-down rate, how often a human corrects or overrides the output, escalation rate, and task-completion rate. Had the eval set already held a question answered by the industrials report, the stale index would have shown up as a score drop within a day. The gap between your offline eval score and these online signals is itself a number to watch: a widening gap means reality has drifted away from your test set. If the only way you learn about a regression is a customer complaint, you don't have observability.
Operational metrics — cost and latency per request
The third stream was sitting in the trace all along: every stage carried a timing, and the model stage carried token counts. Aggregated across requests, those become the operational stream: dollars per request and the latency distribution, read at p50 and p99 rather than the average, because the tail is what a user feels (Cost & Latency covers both). At scale a small regression in either is a large bill or a support queue, and these metrics are also an early-warning channel. A latency spike or a cost jump often surfaces a defect, a retrieval loop or an oversized prompt, before the quality signal moves.
Bad / Good / Great — "how do you debug a wrong answer from last Tuesday?"
Bad — "I'd try to reproduce it in a playground." The deterministic-system reflex, and it proves nothing in either direction: a probabilistic system may answer correctly today after failing last Tuesday, and even a successful reproduction shows you the bad output without showing which stage produced it. An afternoon here teaches you nothing about the request that actually failed.
Good — "I'd pull the logs for that request." The right reflex, go to the recorded data, but service logs record liveness: status codes, latencies, error counts. For a wrong answer they show a request that returned normally in normal time, because operationally nothing failed. A log that carries no answer content cannot say what the system claimed or where the claim came from.
Great — "I'd pull that request's trace and read it stage by stage." Retrieval with chunks and scores, guardrail verdicts, the model call with prompt version and tokens, the groundedness score: each stage with its inputs and outputs, so the failure localizes to a stage instead of to "the model." In the incident above, that read ends at a stale chunk from an old index build; the remedy is a re-index, a scheduled rebuild, and the failing question added to the eval suite so the whole class is caught by the daily run next time.
What to carry into the interview
When an interviewer asks "how do you know it's broken?", or its sharper form, "a user says it gave a wrong answer last Tuesday; walk me through what you do," answer with the shape of this page. Pull the trace of the recorded request and read it stage by stage until the failure localizes. Then name the quality signals, a daily eval score and correction and escalation rates, that would have surfaced the problem before the user did, and the cost and latency you track continuously. Instrumenting quality as well as liveness is what makes the system operable by someone other than you, which is the handover that ends every engagement.
Related: Evaluations (the quality signal), Cost & Latency (the operational metrics), Guardrails (make refusals observable).
Next: Cost & Latency — the two numbers the customer feels before any quality metric.NextCost & Latency