Your 10-page system prompt is a smell

“If your AI agent requires a 10-page system prompt to function correctly, you are likely compensating for model limitations that no longer exist.” This statement, made by the co-founder of a company that handles millions of support tickets each month, resonated widely this week and deserves attention. It highlights a common frustration among teams: as the instruction document expands, the agent continues to misbehave.

A second perspective emphasizes the evolution in this field: prompt engineering was merely the starting point, and the real challenge now lies in context engineering. The key question has shifted from “What should I ask the model?” to “What does the model actually need to know?” The answer is derived from retrieval, memory, tools, and user states, rather than lengthy paragraphs of instructions.

01The prompt is shrinking. The context is the product.

Here’s a practical audit for any overly complex system prompt: each rule in it can be classified as either a true preference (such as tone, format, and boundaries — these should remain in the prompt) or a workaround for missing information (e.g., “the customer’s plan is X,” “inventory updates arrive hourly,” “these three fields are unreliable”). The latter are the indicators of a problem. Each workaround represents context disguised as instruction — information that the model should inherently know, yet is instead documented as a specific behavior it must perform.

Models have significantly improved in their ability to reason from context and intent without needing every behavior explicitly defined. This progress shifts the focus: written descriptions are no longer the most efficient way to convey knowledge. The distinction between a reliable agent and a fragile one increasingly depends on the underlying structures — what gets retrieved at what time, what memories persist across interactions, and which tools are trustworthy for answering specific questions.

02Knowing is engineering. Telling is hoping.

Treating this issue solely as a writing problem is the reason many teams hit a plateau. Determining what a model needs to know — and what it should never have access to — is a critical design decision with significant implications for accuracy, latency, cost, and confidentiality. This requires careful judgment and cannot be simplified into a longer prompt, no matter how well it is phrased.

There is a safety net, and it does not rely on prose either. A common practice emerging among practitioners is to have the model interview you about the expected behavior of the application, then use your responses to scaffold test cases and grading. When context changes subtly — and it always does — evaluations will help identify any discrepancies. Without evaluations, context engineering becomes mere speculation with additional steps.

03We practice it in public

We practice this approach transparently. Our website employs the same principles outwardly, featuring an llms.txt index and agent guidance that inform visiting AI agents about what we do and under what terms. For an agent, this interaction is about retrieval, not persuasion. Internally, we apply the same principles to carefully design the model’s environment.

A valuable question to pose to your team this week is: Of everything in your system prompt, how much is genuine preference, and how much is missing context disguised as instruction?

Sources: the “10-page system prompt” fireside line, the “prompt engineering → context engineering” framing, and the build-eval pattern — all observed on LinkedIn, 2026-09-28/29.