Prompt engineering and structured output

The developer-track version of this subject is “write a good prompt”. The architect version — this exam — is “design a system that produces dependable output”. The difference is discipline: explicit criteria the model can be checked against, examples that teach the boundary cases, a schema that makes the format non-optional, a loop that corrects violations, and an economics layer (batching, multi-pass budgets) that keeps it all affordable. The six subdomains are that sentence, expanded.

01Explicit criteria: the brief, not the vibe

A prompt is a brief for a capable new colleague. That framing survives every exam question in this subdomain because it settles what kind of writing a prompt is: not incantation, not persuasion — a work order. It states the role (who Claude is being: the top-level system parameter, stable across requests), the task (the volatile ask, in the user turn), the scope (what is in, what is explicitly out), and — the architect’s signature — explicit acceptance criteria: what a correct output looks like, stated as checkable conditions rather than adjectives. “Flag risky clauses” is a vibe; “flag clauses that limit liability, transfer data outside the EU, or auto-renew without notice — quote each verbatim — output one JSON object per clause with clause_id, category, severity” is a brief. The exam’s recurring symptom, “too many false positives”, is almost always a criteria problem: the model was never told what not to match.

Structure is the second half of discipline. XML-style tags (<instructions>, <example>, <documents>) give the model unambiguous anchors for what-is-what, and Anthropic’s own prompting guidance leans on them; the stable role lives in the system prompt where it can be prompt-cached, and the volatile material arrives in the user turn; long inputs are chunked and labelled rather than interleaved with instructions. The test for any brief, exam-ready as a one-liner: could a competent human produce the intended output from this brief alone, without asking you anything? If not, the brief is not done — and no amount of model quality will finish it for you.

02Few-shot: examples that teach the boundary

Few-shot prompting is including worked examples in the prompt so the model imitates the pattern rather than inferring it. The exam’s version of “how many and which”: three to five, chosen for diversity — not five variations of the easy case, but a spread that covers the boundary: one clear positive, one clear negative, one genuinely ambiguous case with its resolution. The examples are the specification’s edge, demonstrated. A classification prompt whose examples are all obviously-positive teaches the model that everything is positive; the false-positive rate was written into the examples.

Three design cautions carry marks. Format: examples demonstrate the exact output shape — if the target is JSON, the examples are JSON, formatted precisely as you want it returned (few-shot and structured output compose: the schema enforces, the examples demonstrate). Consistency: conflicting examples — two formats for the same job — produce unstable output worse than no examples. And cost: examples are tokens on every request, which makes them a prompt-caching decision — stable examples belong in the cacheable prefix (the system prompt), volatile demonstrations with the request, so repeated runs reuse the example tokens at a tenth of the price instead of re-paying them. On several current models, non-default temperature and sibling sampling settings are rejected outright in structured contexts — the exam’s reminder that “variety” is not a knob you turn to fix an examples problem.

03Structured output: the schema is the contract

When downstream code consumes the output, prose is a liability and the contract is a schema. The API’s enforcement mechanism is tool use, used sideways: define a tool whose input schema is your desired output shape, force it with tool_choice set to that tool, and the model’s “tool call” is your structured result — JSON conforming to the schema, because the schema is what the tool call had to satisfy. With structured outputs enforced this way, invalid outputs are retried server-side against the schema: the enforcement is contractual, not advisory. (The same idea powers extraction pipelines: the “tool” exists to be called with data, not to do work.)

Schema design is where the marks hide. Declare what is true: required for every field the pipeline actually needs; enums for closed vocabularies (severity: low | medium | high — not free text the model invents adjectives into); types honest and narrow; and where the platform exposes strictness, use it — in strict modes, fields not in the schema are refused rather than tolerated. The classic wrong answers are underspecification (optional everything, “string” everything) and overmachinery (nested arrays three deep where a flat record would do). One more composition: an assistant-side prefill can start the output down the exact format path — and prefill composes with tool use as a formatting aid, though with schema enforcement the format is already the contract.

04Validation, retry and feedback loops

Generation-time enforcement covers shape; the loop covers truth. The examined pattern is validate → feed back → retry: your code checks the output (schema conformance the platform may already guarantee, plus content checks — clause_id exists in the document, dates are real, totals add up, the quote is verbatim), and when a check fails, the retry sends the error back with the failed output so the model corrects itself. That feedback message is a Domain 2 tool error wearing a prompt-engineering coat: what failed, why, and what would pass. “The model returned severity ‘critical’, which is not in the enum; use low, medium or high” — one turn, self-corrected.

The loop needs the same guardrails as any loop. A retry cap (two or three, not “until it works”) so a systematic misunderstanding surfaces instead of spinning; idempotent handling so a re-sent request does not double-charge or double-write; and a failure path that is designed, not discovered — after the cap, route to a human review queue or a conservative default, and log what failed. The exam’s scenario signatures: “keeps failing the same way” → the brief or examples are wrong (fix the spec, not the loop); “occasionally produces one malformed field” → feedback retry; “never malformed but sometimes wrong” → that is not validation’s problem, that is evaluation’s (Domain 5’s calibration, or few-shot’s boundary examples).

05Batch processing

Volume changes the economics, and the API has a dedicated answer: the Message Batches API — asynchronous, at 50% of the synchronous price, with results guaranteed within 24 hours (typically much sooner). Send thousands of requests as one batch; poll or get notified; collect per-request results, each individually succeeded or failed. The exam maps scenarios onto it mechanically: cost priority + can wait = Batches; latency matters = synchronous (with streaming if a waiting human needs progressive output). Two footnotes from the sibling domain: batches cannot carry every feature (fast mode is unavailable through it, current as of September 2026), and a batch is capped at 256 MB where a single message is 32 MB.

Batch design is throughput design. Chunk large corpora into requests that each fit comfortably; keep the stable prefix (role, criteria, examples) byte-identical across requests so prompt caching applies inside the batch too — cache reads at a tenth of input cost compose with the batch’s half-price. Watch rate limits (requests, input and output tokens per minute — tiered), respect 429s with backoff, and design the pipeline to be resumable: a batch that dies at request 4,000 of 10,000 should resume from 4,001, not restart — which is the same session-state lesson as Domain 1, at pipeline scale. And keep a holdout: a sample of batch results human-checked each run, because half-price output drifting quietly wrong is the most expensive cheap thing in this domain.

06Multi-instance and multi-pass review

The last subdomain raises the reliability ceiling by spending more of it. Multi-instance: run the same step two or more times — with the same prompt (consensus by majority vote) or with different framings (diverse independent assessments) — and aggregate: majority vote for classification, intersection for high-precision extraction, union with deduplication for recall-critical sweeps. Multi-pass: the same content through sequential passes with different jobs — extract, then verify; draft, then critique; translate, then review against the source. Generator–critic is the named pattern: a second instance (or a second prompt) whose only job is to find flaws in the first one’s output, feeding corrections back — Domain 4’s feedback loop, staffed.

The architect’s questions are exactly the exam’s: when is the multiplier justified, and what does aggregation do to the error profile? Multiplying instances multiplies cost linearly but does not fix a shared blind spot — three readers of the same ambiguous clause all misread it the same way; the fix for shared bias is diverse prompts or a different model, not more copies. Voting trades one kind of error for another: majority vote suppresses occasional misses (improves precision-or-recall one way) while burying correct minority answers (the other way) — choose the aggregation rule to match which error hurts the scenario more. And verification passes are cheap insurance relative to the blast radius of what they gate: the bigger the downstream action (Domain 1’s escalation ladder again), the more passes the economics justify.

07The traps

• —“Ask for valid JSON in the prompt — that’s structured output.” Asking is advising; a forced schema tool is enforcing. Prompt-asked JSON fails at the exact rate that matters: occasionally, in production, silently.
• —“More examples is always better.” Three diverse boundary-teaching examples beat twenty variations of the easy case — and cost a twentieth of the tokens on every cached-miss.
• —“It failed validation, so retry the identical request.” Without the failure fed back, the model has nothing new to work with — that is a thrash loop with extra steps. Feed the error, or fix the brief.
• —“Batch everything — it’s half price.” Batches buy 50% with a 24-hour ceiling. A waiting human or a blocking pipeline step makes that trade absurd; latency-sensitive work is synchronous, streamed.
• —“Run it three times and take the majority — reliability solved.” Voting suppresses occasional errors and preserves systematic ones. If the brief is wrong, the vote is unanimous.
• —“Tune the temperature for consistency.” On several current models, non-default sampling settings are rejected outright in structured contexts — and consistency was never a sampling knob: it is criteria, examples and schemas.