Model routing

01What it means

Model routing is the practice of directing each piece of work to the model that fits it: small, fast, cheap models for mechanical calls; frontier reasoning models for hard analysis; and explicit thresholds for when a task escalates between them. It treats model choice as an engineering decision with a cost, latency, and quality budget — not a brand preference set once at project kickoff.

Two shifts made it standard practice. Model families now ship in tiers with wide price gaps between them, so sending everything to the flagship is the most expensive way to build — and falling token prices keep widening the spread between doing it right and doing it lazy. And routing can be confidence-gated: the answer tells you what, the probability tells you whether to act — auto-refund at 0.98, hold for a human at 0.72. Deterministic code owns the workflow, the model supplies judgment, and the uncertain cases go to a person.

The routing table is also a quality instrument: escalation paths to stronger models — or to humans — are how hard cases get handled instead of guessed at. Routing done well is invisible; routing done badly shows up as either surprising invoices or surprising mistakes.

02Why it matters for buyers

• —Cost: routing mechanical work to cheap model tiers is the single biggest cost lever in an AI system — bigger than prompt tuning, vendor haggling, or caching.
• —Quality: ask where work escalates. A vendor who can name the path — weak model to strong model, low confidence to human — has designed for the hard cases.
• —Ask for the routing policy, not the model name: "which model, for which task, with what escalation threshold" is a concrete, checkable answer; "we use the best model" is not.

03Common questions