Are skyrocketing AI bills eating into your budget? Learn why using frontier models for simple coding and utility tasks is like driving a supercar to buy milk, and how smart model routing can cut compute costs by up to 10x.
If you just need to pop down to the corner shop for a pint of milk, you wouldn't fire up a V12 Ferrari, warm up the engine, and burn through high-octane fuel for a two-minute trip. A simple bicycle or a quick walk gets the job done without the astronomical running costs.
Yet, across the tech industry, engineering teams are doing the exact equivalent with AI. Companies are watching their monthly AI bills explode into eye-watering territory because they route every single prompt, from complex system design down to minor spell checks, straight to expensive frontier models.
The Problem: Over-Engineering Every Prompt
Frontier models are brilliant, but using them for basic tasks like parsing JSON, fetching context, or simple syntax checks is financial suicide. When I wrote about AI Performance Optimisation, the message was clear: speed and cost efficiency are non-negotiable for sustainable AI engineering.
You don't need a massive multi-billion parameter model to run grammar checks or post a quick status update. What you actually need is an intelligent routing system that looks at the task at hand and dispatches it to the most cost-effective model or service available.
How I Handled This in The Foundry
In my own workflow tool, The Foundry, I built a tiered model routing framework to keep API spending grounded while keeping output quality extremely high. The architecture relies on separation of concerns across different model tiers:
Tier 1 (Orchestrator & Reviewer): High-capability models handle system architecture, complex reasoning, and final code reviews.
Tier 2 (The Builders): Faster, targeted models execute standard coding tasks based on strict specifications.
Tier 3 (The Helpers): Lightweight or local open-source models handle posting updates, gathering context, or running basic linting passes.
By shifting day-to-day execution to local compute or smaller API tiers, total compute costs can drop by almost 10x without sacrificing final quality.
Spec-Driven Governance: Making Small Models Reliable
Can you trust a smaller model like Qwen 3.6 24B or Claude Haiku to write reliable code? Absolutely, provided you have spec-driven development and governance rules in place. The spec gives the model precise boundaries, while governance enforces coding standards.
Once the lower-tier model completes the heavy lifting, an orchestrator agent or peer model reviews the code against the spec. If you run a solid local model, it can even review its own output before sending it up the chain.
The Takeaway
Stop driving your Ferrari to the corner shop. By decoupling task planning from grunt execution, you can leverage local compute and smaller models for standard jobs, reserving top-tier models strictly for orchestration and review. Your team stays fast, your code stays clean, and your cloud provider stops taking all your budget.