Context management and reliability

The API is stateless: every request resends everything — the whole conversation, every tool result, every instruction — and the model sees only what this request contains. A 200K-token window sounds unfillable until an afternoon of agentic work fills it: tool outputs are enormous, dead ends are still in there, and nothing leaves unless you manage it. Context management is the named set of levers; reliability is what the levers protect.

01The named levers — and when each applies

The exam expects you to match symptom to lever, and the levers have names. Compaction: replace the older history with a summary and continue — the session stays one conversation, at one-tenth the size. Context editing: clear the oldest tool results (keeping the most recent tool calls and their results, and older thinking is summarised away by design) — surgical relief for tool-output bloat, without rewriting the whole thread. The memory tool: durable notes stored as files under /memories, written and read on demand — knowledge that must outlive sessions: user preferences, decisions, the map of the codebase. Delegation: hand work to a subagent whose noise never reaches the parent window — isolation, Domain 1’s first reason. And prompt caching belongs in the same breath for a different reason: it does not shrink anything, it makes the stable prefix cheap — write once at a premium, read at a tenth on every subsequent call, five-minute TTL by default with a one-hour option.

The vocabulary that organises them — and a known exam distinction: pruning clears content after it arrives (compaction, context editing); isolation stops it arriving at all (subagents, tool search). Two different cures for the same swelling. And the memory tool is neither: it is durability — persistence across sessions, not relief inside one. When a scenario hands you a symptom, run the framework: the session is long and must continue as one conversation → compact; the bloat is old tool output specifically → context editing; the knowledge must survive to tomorrow → memory tool; the noise should never have reached this window → delegate; the window is fine but the bill is not → caching. One line each, and every question in this subdomain is one of the five.

02Escalation and ambiguity resolution

Long-running agents meet questions they cannot settle alone, and the exam’s phrase for the design is an escalation ladder: the agent resolves what it can, defers what it must, and a human resolves what climbs. The triggers are the design decisions: blast radius (an action that cannot be undone climbs), ambiguity the agent can detect (two readings of a requirement, both plausible), confidence below threshold (next section), and policy (categories of decision an organisation has decided humans make). Each rung buys safety and costs latency — the trade is explicit, and the scenario always tells you which side the business is weight-sensitive on.

Ambiguity resolution is the rung below: the decision between asking and deciding. The rule the exam rewards: ask when the cost of asking is lower than the expected cost of acting wrong — a one-line clarification versus a migration on the wrong table. But asking is not free either (interrupted flows, human fatigue, agents that ask about everything), so the craft is making asks rare and cheap: resolve what the brief settles; state assumptions the user can correct in passing when stakes are low; batch questions rather than drip them; and reserve the true interrupt for the irreversible. An agent that asks nothing is dangerous; an agent that asks everything is unusable — the exam’s scenarios are calibrated between those poles.

03Error propagation in multi-agent systems

In Domain 1 the coordinator received reports; here the exam asks what happens when a report is bad — or never comes. The design question is always the same: when a worker fails, who learns what, and what happens to the mission? The propagation patterns are three postures. Fail fast: the first worker failure aborts the run — right when partial results are worse than none (a financial report from three sources where one is wrong is a wrong report). Retry and continue: transient failures (a tool timeout, a rate limit) are retried with backoff and the mission proceeds — right when units are independent and the odds favour recovery. Degrade gracefully: the mission completes minus the failed part, with the gap declared in the result — right when 90% coverage today beats 100% next week, provided the output says which 10% is missing.

Three propagation rules keep the patterns honest. Surface, never swallow: a worker that catches its own errors and returns “done, with caveats” poisons every downstream decision — the coordinator must see failures as failures, structured (Domain 2’s error contract, agent-to-agent). Propagate with context: “worker 3 failed: the payments API returned 429 for ten minutes; partial results cover services A–F” teaches the coordinator something; “error” does not. And mark confidence at the boundary: the coordinator’s report must carry what it is built on — which findings are from complete passes, which from degraded ones. The exam’s symptom phrases map to broken rules: “the coordinator reported success with three workers failed” = swallowing; “we could not tell what went wrong” = context-free propagation; “the final answer cited the failed source” = no confidence marking at the boundary.

04Context in large codebase exploration

You cannot read a repository into a window, and the exam’s position is that you should not try: exploration is agentic search, not bulk ingestion. The loop from Domain 1, pointed at a codebase: form a hypothesis about where the answer lives (Glob by name pattern, Grep by content), read narrowly (Read with offset and limit on the hits), refine the hypothesis, repeat. Each step costs a fraction of a bulk read, and the dead ends — the missed grep, the wrong directory — are information instead of baggage. This is also why read-only exploration subagents exist (isolation again): let the noise happen in a window you throw away, and keep the distilled map in the window you keep.

The architect’s choices in a scenario: targeted search over bulk reading wherever the question is locatable (“where is X handled?” never needs the whole file tree); narrow reads over whole-file reads for big files (offset and limit are context budget lines); delegation when exploration volume is large or disposable; compaction or context editing when a session’s accumulated exploration is crowding out the work; and the memory tool (or CLAUDE.md, or a notes file) when the map itself is the durable asset — the layout you learned today is the thing tomorrow’s session should not have to relearn. The trap answers are always bulk: “read all files then decide”, “increase the context window”, “paste the repository into the prompt”.

05Human review and confidence calibration

Domain 1 put humans at the top of the escalation ladder; this subdomain is the engineering of that rung. The design has three parts. Gating: which outputs route to review — by blast radius (everything that writes, sends, deletes, or spends), by confidence (below a threshold, next paragraph), and by sampling (a random slice of the confident stream, audited, because quiet drift is the failure mode confidence gates cannot see). Presentation: the reviewer sees the proposal with its evidence — the diff, the sources, the chain of decisions — enough to judge quickly without re-doing the work; the moment reviewing costs as much as doing, the rung was mispriced. Feedback: every human verdict flows back — into the prompt, the examples, the thresholds — because a review queue that only catches and never teaches is a subsidy, not a control.

Confidence calibration is the subdomain’s technical core, and the exam’s line is blunt: a model’s self-reported confidence is not, by itself, a reliable number — it must be calibrated against measured accuracy on your task, with your data, via evaluation sets. Calibration is the mapping from stated confidence to observed correctness: run a labelled set, bucket outputs by stated confidence, measure precision per bucket, and set the threshold where the measured trade (misses versus review load) matches the business’s tolerance. Once calibrated, the number earns its job — auto-approve the high band for low-stakes actions, review the middle, block or redesign the low. What calibration forbids is the naive pattern: “the model says 95% confident, so we trust it” — an unmeasured number gating an irreversible action is the exam’s wrong answer with a suit on.

06Provenance and uncertainty in multi-source synthesis

The final subdomain is about output you can audit: when an answer synthesises multiple sources — documents, tool results, worker reports — the design must carry where each piece came from and how sure it is. Provenance is source-tracking at the claim level: findings cite their sources (the document and passage, the tool result, the worker ID), structured rather than rhetorical — a findings array with a source field, not a closing paragraph that mentions some files. The mechanism composes with everything upstream: schemas can require source fields (Domain 4), workers can be required to return evidence with every claim (Domain 1’s interface contracts), and citations can quote or link rather than paraphrase, because paraphrase is where provenance quietly dies.

Uncertainty is provenance’s honest sibling. Sources conflict, evidence is thin, retrieval came back empty — and the synthesis must say so, in the output’s structure, not its adjectives: a field for confidence, a list of open conflicts, “not found” as a first-class finding rather than a guess dressed as one. The exam’s named failure is laundering uncertainty: worker A is unsure, the coordinator summarises A’s hedge into a declarative sentence, and the final report asserts what was never established — each hop of summarisation stripping the qualifiers until a maybe arrives as a fact. The design cures are structural: confidence travels with the claim through every interface; synthesis rules state when to present disagreement as disagreement; and the report reserves assertion for what the evidence supports, with the rest marked. That last habit is not modesty — it is the difference between an answer you can defend line-by-line and one you can only hope about.

07The traps

• —“The session is huge — increase the context window.” The window is a given; the levers are named. Compact, edit, isolate, or persist — “get a bigger window” is never the exam answer.
• —“Use the memory tool to fix the long session.” The memory tool is durability across sessions, not relief inside one. A session that must continue as itself wants compaction or context editing.
• —“Cache to make the long session cheaper.” Caching makes a stable prefix cheap to re-read; it shrinks nothing. Bloat is a management problem, not a pricing problem.
• —“The coordinator saw every worker succeed — the report is right.” Success-of-call is not correctness-of-result; mark confidence at boundaries and sample-audit the confident stream.
• —“The model says 99% confident.” Unmeasured confidence is a number, not a measurement. Calibrate against a labelled set before any of it gates an irreversible action.
• —“Read the repository first so it has full context.” Bulk reading loses to agentic search on cost, focus and refresh rate — and the dead ends become baggage that crowds out the actual work.
• —“Synthesis sounded confident, so we shipped it.” Sound is not provenance. Claims cite sources; conflicts are listed; maybes arrive as maybes — or the report is a hope with formatting.