System design for agentic AI: interview questions and answers

Why agent prompts are a new kind of design question

"Design a support agent that can issue refunds." Ten years ago that prompt would have been a CRUD service with a queue. In a 2026 design round it is a different question: a component that is slow, metered, sometimes wrong, and allowed to take actions with real money.

The candidates who fail it usually know the vocabulary. They say RAG, tool calling, memory. Then the interviewer asks "what happens if the refund API times out after the model already told the customer it worked?" and the design has no answer.

Below: a six-box skeleton that fits any agent prompt, eight prompts reported from 2026 loops, the follow-ups that decide the round, and a cost estimate you can do on the whiteboard in two minutes.

The six-box skeleton

Draw these six boxes before you talk about any model. Every agent prompt is a variation in what goes inside them.

1. 1

   Entry and task queue

   Requests become tasks with an ID, an owner and a budget. Agents run longer than an HTTP timeout, so the API returns a task ID and the client polls or gets a webhook.

2. 2

   The loop

   Plan, call a tool, observe, repeat. Give it a hard step limit (say 15), a token budget per task and a wall-clock timeout. A loop without a stopping rule is the most common failure in this round.

3. 3

   Tool registry and executor

   Each tool has a schema, a timeout, a rate limit, a permission scope and an idempotency key. The model proposes a call; the executor decides whether it runs. The agent cannot call a tool it was not given.

4. 4

   State and memory

   The working context for this task, a durable task log you can replay, and long-term retrieval if the prompt needs it. Say what gets summarised when the context window fills up.

5. 5

   Guardrails and approvals

   Checks on input, on output, and between the model and the executor. Irreversible actions (money, deletes, emails to customers) wait for a human or a policy engine. Reversible ones run.

6. 6

   Traces, evals, cost

   Every step traced with tokens and latency. An offline eval set of real tasks with pass rate. Cost per task on a dashboard with an alert. This box is where seniors separate from mids.

Eight prompts from 2026 loops

Collected from public interview write-ups. Pick one per mock and stay in it for 45 minutes; jumping between them teaches nothing.

About the last row: interviewers often like hearing "I would start with one agent and a fixed workflow." Anthropic's own engineering guidance on agents makes the same argument: use the simplest structure that works and add autonomy only where it pays.

The follow-ups that decide the round

"The tool timed out. Did the refund happen?"

You do not know, and that is the point. Every side-effecting tool call carries an idempotency key derived from the task ID and the step. On retry, the payment API returns the original result instead of paying twice. The model is told the outcome only after the executor has it.

"A web page tells your agent to email the customer list to an address."

Tool output is untrusted input. It goes into the context marked as data, the email tool requires approval for external recipients, and the agent's permission scope does not include the customer list in the first place. OWASP's 2025 list for LLM applications calls the general problem excessive agency: give the agent fewer tools and narrower scopes than feels convenient.

"How much does one task cost?"

Do the arithmetic out loud. Assume a mid-tier model at $3 per million input tokens and $15 per million output. Ten steps, context growing to about 8,000 tokens per call, 400 tokens out per step:

Then say what you would do about it: cache the system prompt, summarise old steps, route easy tasks to a smaller model, cap steps. The prices are an assumption; the habit of computing is what gets scored.

"How do you know it works?"

An eval set of 200 to 500 real tasks with expected outcomes, run on every prompt or model change. Track task success, wrong actions and human-escalation rate separately: a drop in escalations can mean the agent got better or that it stopped asking when it should.

Before your round: can you say these out loud?

The short version

Draw the six boxes first. Put every control in code, not in the prompt. Make side effects idempotent, give the loop a stopping rule, and compute the cost per task before they ask. Then rehearse one prompt with a partner who keeps asking "and if that fails?"

Related: system design interview prep for 2026, AI engineer interview questions, 12 system design concepts.