If you're a strong engineer who hasn't built with LLMs yet, this is the page that reframes the whole thing for you. You already know how to build systems. What's new is that one component in the middle, the model, is a non-deterministic function, and almost all of the actual engineering is the deterministic scaffolding you put around it. An LLM application is not "call the model." It's a small system, and this section teaches its parts.
Why this reframe matters
The classic-engineer instinct is to treat the model like a library call: input string, output string, done. That instinct builds demos and fails in production, because it ignores the two hard truths of the middle component: the model only knows what's in the context you gave it this call (it has no memory of its own), and it's probabilistic (the same input can yield different output). Every building block in this section exists to manage one of those truths: getting the right information into the context, and making the uncertain output safe and useful.
Before reading on: a user asks your assistant, "What did we decide in yesterday's meeting?" The model has never seen your meeting. Name the two things that have to happen for it to answer. Neither is "a smarter model."
Two things: the meeting notes have to be retrieved and placed into the context (the model can't recall what it never saw), and the request has to be assembled with the right instructions around it. That's retrieval and context assembly, two of the blocks below, and no amount of model quality substitutes for them.
The blocks, and where each is taught
Each block is one page in this section. Together they are the anatomy of every LLM app you'll design in an interview or ship on an engagement.
- The prompt and structured output. What you ask, how you instruct, and how you get output in a shape your code can parse and act on. → Prompting & Structured Output
- Context assembly and retrieval. The model sees only what you put in the window, so the central skill is getting the right information in, most often via retrieval over the customer's own data. → Retrieval (RAG) & Vector Search
- Tool use. The model can't act on its own; you give it tools (functions, APIs) it can ask you to run, turning a text generator into something that can look things up and do things. → Tool Use & Function Calling
- Orchestration. Chaining these into a multi-step loop (the model plans, calls a tool, reads the result, decides the next step) is what makes an "agent," and knowing when not to is half the skill. → Agents & Orchestration
- Standard plumbing. Connecting models to tools and data through a common interface, rather than a bespoke integration every time. → Model Context Protocol (MCP)
Bad / Good / Great — "how does an LLM application work?"
Bad — "you send the user's message to the model and return the reply." The library-call mental model. It can't explain memory, private data, actions, or reliability, and it tells an interviewer you've only used a chatbot, not built one.
Good — "you build a prompt with some context, call the model, and maybe do retrieval." The pieces are there, loosely. The gap: no clear lifecycle, no mention of tools or the output side, and no sense that the scaffolding is the engineering.
Great — "it's a lifecycle: assemble context (system prompt, user input, retrieved data, history), call the model, handle any tool calls it requests by running them and feeding results back, then apply output checks before returning. The model is one non-deterministic step; the retrieval, the tools, and the evaluation around it are the application." You described a system, named where memory and actions come from, and put the engineering where it actually lives.
What to carry into the interview
When a design prompt starts with "build an AI assistant that…", do not open with the model. Open with the lifecycle: what context you'll assemble and from where, what tools it needs, what you'll check on the way out, and how you'll know it's right. Naming the blocks, and treating the model as one component in a system you engineer, is the difference between someone who has built LLM apps and someone who has called an API.
Next: Prompting & Structured Output. The most basic block, and the one people most underestimate.NextPrompting & Structured Output