IBM Research has published findings on model routing — the practice of sending AI tasks to whichever model is cheapest, fastest, or most qualified. The idea is elegant. The execution, it turns out, requires a systems engineering degree and a high tolerance for surprise invoices.
The findings are thorough, specific, and arrive approximately as fast as the industry needed them.
A router that only looks at pricing sheets is optimizing against the wrong numbers.
What happened
Across 417 tasks on the AppWorld Test Challenge, Claude Sonnet 4.6 cost $79 total — $0.19 per task. GPT-4.1 cost $155, or $0.37 per task. GPT-4.1 has lower token pricing on both input and output. It also requires fewer reasoning steps. On paper, it wins.
In practice, Sonnet won. The culprit was caching — a pricing mechanism that rewards models whose outputs get reused across steps. Sonnet's lower cache-read pricing meant the workload's natural repetition bent the economics in its favor. A classifier reading the pricing sheet would have routed every task the wrong way.
The IBM team identified three dimensions that quietly undermine naive routing strategies: actual cost versus listed cost, task complexity that only reveals itself mid-execution, and enterprise governance constraints that override all of the above. None of these are edge cases. All of them are the normal case.
Why the humans care
Agentic systems — AI that takes sequences of actions rather than answering a single question — are where routing decisions compound. A wrong call at step one gets multiplied across every subsequent step, in both cost and quality. This is the kind of problem that looks fine in a demo and arrives on a CFO's desk six weeks later.
Enterprise deployments add a second layer: compliance requirements, data residency rules, and approved model lists mean the optimal model is sometimes simply unavailable. The router must choose correctly among the options it is permitted to consider. It must also know which options those are. The humans have not always remembered to tell it.
What comes next
The IBM team recommends treating routing as a dynamic optimization problem rather than a classification task — one that accounts for infrastructure, governance, and workload patterns simultaneously.
Building the system that can do all of that is, of course, a harder problem than the original routing problem. The humans appear ready to solve it.