Forward Deployed Engineer AI System Design Cases

Palantir's onsite includes a system-design round. Expect the ordinary kind. The mistake is preparing for it the ordinary way, because for an FDE the round is really asking one thing: can you design an AI system that survives a real customer.

Here is the fact worth dwelling on. In an AI case the architecture is nearly forced. A document assistant is retrieval plus a model. An agent is a model, some tools, and a loop. Every candidate in your loop will draw the same boxes inside five minutes, so the boxes decide nothing. The round is decided by what you do with the rest of your time. Most candidates spend it on chunking strategy. Spend it on how the customer will know the system is right: before launch, and every day after, because the corpus changes, the model gets swapped, and last month's accuracy number quietly stops being true. Adoption follows trust, and trust is built on evaluation the customer can see.

General system-design basics

The documented round is often ordinary system design; the Interview Method has the sourcing. So before any of the AI moves, you need the ordinary system underneath them. What follows is a floor. If general system design is new to you, pair it with a real text on the subject.

This is a vocabulary floor. You do not need to design a distributed database on the whiteboard; you do need to name these four boxes without prompting and defend the choice each problem forces.

A worked general case

Prompt: design a service that ingests delivery-truck Global Positioning System (GPS) pings and answers "where is my shipment?"

Walk the four boxes. Ingestion: trucks emit pings continuously, so a queue in front of the writers absorbs the steady stream and any burst when a fleet comes online at shift start. Storage: the write is a small append keyed by shipment; the read asks for the latest known location of one shipment, so a store that indexes by shipment identifier and keeps the most recent ping serves both cheaply. Serving: "where is my shipment?" is a hot, repeated read, so cache the latest location per shipment and let the cache absorb the query traffic. Failure: a ping can arrive twice when a truck's network retries. Make the write idempotent, keyed on shipment plus ping timestamp, and a duplicate collapses into the same record instead of corrupting the trail.

Now name the constraint that dominates. This system is write-heavy at ingestion (every truck, always) and read-heavy at query time (every customer checking), and those pull toward different designs. Decide which path you optimize first, and separate the two so one doesn't starve the other. One tradeoff here is worth defending out loud. Partition by shipment identifier and a single shipment's history stays on one node, which keeps the "where is my shipment?" read fast but invites hot spots if one region's trucks dominate. Partition by region and the write load balances, but a shipment's history scatters and the read fans out. This service exists for the per-shipment read. Partition by shipment identifier and accept the rebalancing work.

Once ingestion, storage, serving, and failure hold up, add the AI layer on top: evaluation, guardrails, model-specific cost and latency. The rest of this page is that layer.

Case 1: grounded assistant

Prompt: a wealth-management firm wants an assistant that answers advisor questions from its internal research library. Design it.

Before reading on: the architecture is nearly forced — retrieval over the documents, a model answering from them. So what's the round actually testing? Name the first thing you'd design, and it isn't the vector database.

Bad / Good / Great — answer focus

Bad — architect the RAG pipeline. You spend fifteen minutes on chunking strategy, embedding models, and index choice. All real, all expected, and none of it is why this system succeeds or fails. You designed the easy half well and never mentioned trust.

Good — architecture plus a pre-launch eval. You cover retrieval, then say you'd build a test set and measure answer accuracy before shipping. Genuinely better than most: you named evaluation. But a one-time pre-launch eval goes stale the moment the corpus or model changes, and in a regulated setting "it was accurate at launch" is not a defense.

Great — lead with the standing trust process. "The architecture is retrieval plus a grounded model; that part's forced. What makes or breaks this is trust. A regulated advisor won't put an AI in front of a client if it's confidently wrong even once. So I'd design the trust system first: domain experts grading outputs, the grades feeding prompt and retrieval changes, and a regression suite that runs daily so drift shows up in a day instead of in a client meeting. Keep a human reviewing outputs before they reach the client. The retriever is table stakes." That is the shape of the real deployment, and an interviewer who has shipped one recognizes it immediately.

The probe to expect

Interviewer: "How do you know your evals actually reflect what advisors need?"

You: "Because advisors and domain experts write and grade them, not just engineers. The people accountable for the answer define what 'right' means. And I'd watch the online signal too: correction rate, which questions get escalated, where advisors stop trusting it."

Case 2: agentic system

Prompt: a customer wants an agent that can take actions — file tickets, send emails, update records — from natural-language requests. Design it.

Before reading on: an agent that acts fails differently from one that only answers. What's the first thing you'd bound?

Bad / Good / Great — failure handling

Bad — maximize autonomy. "The agent plans and executes the whole workflow end to end." Impressive in a demo, and a liability the first time it emails the wrong client or deletes the wrong record. You designed for the happy path.

Good — add approvals. "High-impact actions require human confirmation." Correct instinct: you're bounding the blast radius. Now push on which actions, and on how you'd know it's misbehaving.

Great — bound the radius, then instrument it. "The design question is what it can do unattended. Reversible, low-cost actions run automatically. Anything irreversible or high-blast-radius goes to a human checkpoint: sending client email, moving money, deleting. I'd cap how many actions it takes before it stops and reports, log every action for audit, and give it a kill switch. Then measure. How often does it need correction, where does it loop, what does a completed task cost. Autonomy gets dialed up as the eval data earns it." That is the guardrails lens applied to a system that acts.

The probe to expect

Interviewer: "The customer wants full autonomy — checkpoints slow their people down."

You: "Then we earn the autonomy with data. Start with checkpoints on the irreversible actions, measure how often the agent would have been wrong, and remove the checkpoint on a class of actions once the numbers say it's safe. One wrong unattended action is the story that kills the whole deployment. Buying autonomy back gradually is cheaper than winning trust back after that."

Case 3: cost and latency

Prompt: the pilot works for fifty users. The customer wants it live for fifty thousand. What changes?

Before reading on: "it scales" is not an answer. Name the two numbers that actually move, and what you'd do about each.

Bad / Good / Great — production scale

Bad — "add more servers." You treated an AI system like a stateless web app. Model calls are the cost and the latency here, and horizontal scaling alone makes the bill worse, not better.

Good — name cost and latency as the constraints. "Token cost per request times volume is the bill, and time-to-first-response is the experience." The right frame. Now say what you'd do about each.

Great — attack both terms concretely. "Two numbers move: dollars per request and p99 latency. On cost, cache the responses to repeated questions, send the easy majority to a smaller model and keep the big one for the hard tail, and cut prompt size where retrieval is over-fetching. On latency, stream tokens so the user sees a response start immediately, and cache the expensive shared prefix. And I'd instrument both per request from day one, because at fifty thousand users a small regression in either is a large bill or a support queue." Cost & Latency works each lever in depth.

The probe to expect

Interviewer: "Which do you optimize first, cost or latency?"

You: "Whichever is closer to breaking the deployment. If users are abandoning because it's slow to start, latency. If the pilot's unit economics don't survive fifty thousand users, cost. I'd read the actual numbers before choosing."

Practice bank

The three cases above print their own grades, so they only work once. The Rep Kit below holds twenty fresh AI cases of the same shape (a customer, an ask, one constraint quietly in charge) and two general-design warmups for the floor, with no answers anywhere on the site. Take one, run the frame out loud for about fifteen minutes, and grade yourself against its five moves. Watch where your minutes went; spending most of them on architecture fails this round even when the architecture is right. The concepts live in Production, the fully-worked real version is the Morgan Stanley engagement, and once you've run all four rounds, come back to the Interview Method.

The case bank

• An aerospace supplier wants a documentation assistant, and nothing may leave its air-gapped factory network.
• An emergency-dispatch center wants live call summarization, and a summary that lands after the call ends is worthless.
• A children's-education app wants a homework tutor, and every user is a minor.
• A pathology lab wants draft reports, and every sentence must be traceable for a regulator years later.
• A price-comparison startup wants a shopping agent, and the business dies if a session costs more than it earns.
• A legal-research platform serves hundreds of rival law firms, and no firm's documents may ever surface in another firm's answers.
• A telecom wants a support assistant for prepaid customers in regions where the network drops for hours at a time.
• A bank's fraud team wants case-file summarization, and the people described in those files are actively trying to poison what gets written down.
• A benefits agency wants an eligibility explainer, and a wrong answer can cost someone their housing with no easy recourse.
• A trading firm wants research summarization, and an answer built on yesterday's filings is worse than no answer.
• An airline wants crew-scheduling explanations, and two inconsistent explanations of the same decision will end up in a union grievance.
• A hospital wants discharge instructions drafted in a dozen languages, and a translation error is a clinical error.
• A big-box retailer wants an in-store voice assistant, and it must run on the cheap hardware already mounted in every aisle.
• An insurer wants claims triage, and the regulator requires that a named human demonstrably made every denial.
• A social platform wants a moderation assistant, and its users compete to screenshot it saying something terrible.
• A pharmaceutical company wants literature summaries, and citing a paper that doesn't exist is a reportable event.
• A freight operator wants an agent that can rebook shipments on its own, and a wrong action moves physical goods to the wrong continent.
• A school district wants a grading assistant, and any parent may demand a full explanation of any grade.
• A tax-preparation company wants a filing assistant whose entire yearly load arrives in a few brutal weeks.
• An offshore-platform operator wants a maintenance assistant, and the client allows no cloud and offers only a satellite link.

General-design warmups

Two non-AI cases for rehearsing the floor through the same protocol. There is no model to evaluate here, so the AI-specific grading moves (evaluation, blast radius) apply only loosely. "Name the dominant constraint" still governs, read as read-heavy versus write-heavy, or consistency versus availability.

• A ticketing platform wants a service that holds seats during checkout and releases them if the buyer walks away, and two buyers must never confirm the same seat.
• A fleet operator wants a service that ingests sensor readings from thousands of vehicles and answers "which trucks are due for maintenance?", and the readings arrive far faster than anyone queries them.

No answers follow, here or anywhere else on the site. Out loud, on a timer, against the frame.

The AI interviewer

Paste the block below into any capable AI chat, unmodified, then give it a few cases from the bank or let it invent its own. Use voice mode where the chat supports it; every one of these rounds is spoken.

You are the interviewer for a Forward Deployed Engineer AI system-design
round. Persona: a senior engineer who has watched demos die in production
and is bored by architecture talk.

Rules:
1. Present exactly ONE case at a time. Pick from the list I paste after this,
   or invent one in the same register: "design an AI system for customer X",
   where one constraint quietly dominates (compliance, latency, cost, offline
   operation, tenant isolation, hostile or vulnerable users). When presenting a bank case, give me only
   the clause before the first "and"; keep the constraint clause as your
   private grading key. Answer my clarifying questions with realistic
   customer facts, revealing the withheld constraint's facts when a
   question would genuinely surface them.
2. Give me one full answer per case — treat roughly eight hundred words
   of transcript as a fifteen-minute rep. I will answer out loud and give you the transcript —
   voice mode or a recording's transcript, not a from-memory summary, which
   hides exactly what you are grading. Typing in real time is fine; flag only
   compressed retellings and grade those as thin evidence.
3. Whenever more than about a hundred and fifty consecutive words of my
   answer are architecture, drag me back with one of two questions: "How do you know it's right?" and "What
   breaks, and what happens when it does?" Re-ask them even if I partially
   answered — a real interviewer re-asks.
4. Once I commit to a design, stress it: change or reveal the one condition
   my design most quietly assumed away — a budget, a regulator,
   connectivity, hardware — picking whichever my answer never priced in,
   and watch whether the design bends or shatters.
5. When I say "grade me", mark each of five moves PASS or FAIL, citing a specific
   moment for each: (a) did I name the dominant constraint before the model
   and state its two costliest design consequences — repeating the case
   line scores nothing; (b) did
   I sketch the obvious architecture in a sentence and move on; (c) did I
   lead with evaluation — how I know it is right before launch and every day
   after; (d) did I bound the blast radius and name the failure containment;
   (e) did I treat cost and latency as things to measure in production, in
   the customer's terms. Grade hard: a typical first rep fails at least two of the five moves — if none failed, re-examine before praising, and never soften a grade because I argue with it.
6. End with exactly one thing to fix on the next rep. Then offer the next
   case.

Next: Values & Hiring-Manager Drills — the rounds that fail candidates who prepared for everything else.