Somewhere in the last eighteen months, "what's our AI spend actually going toward" stopped being a question finance could shrug off. According to the FinOps Foundation's sixth annual State of FinOps survey — 1,192 practitioners, more than $83 billion in tracked annual cloud spend — 98% of organizations are now managing AI spend as part of their FinOps scope, up from just 31% two years earlier. That's about as fast as any cost category has ever entered the boardroom conversation.

Here's the thing though: "managing" it and "controlling" it are two different skills, and most teams are still building the second one. The same survey found AI cost management is now the single most in-demand skillset FinOps teams are trying to hire or train for — ahead of cloud optimization, which has been the dominant skill for the better part of a decade, according to reporting from CIO Dive.

If you've ever built a vendor scorecard or sat on a buying committee, this should feel familiar. The tools below aren't exotic. They're the AI-era equivalent of reading the contract before you sign it.

Way 1: Cache What You're Already Paying to Re-Send

Every time a conversation includes the same system prompt, the same tool definitions, or the same reference document as the turn before it, most teams are paying full price to process that content again — even though nothing about it changed. That's the single most wasteful default in a typical AI deployment, and it's also the easiest one to fix.

Prompt caching works by marking the stable part of a prompt — system instructions, tool schemas, a long reference document — so the model can reuse its previous read of that content instead of reprocessing it. Anthropic's implementation prices a cache write at a modest premium (1.25x to 2x standard input, depending on how long you want it held), but every subsequent read within that window costs a fraction of the original price. OpenAI runs automatic caching by default on prompts over roughly 1,000 tokens, no configuration required.

This is the rare cost lever with basically no downside: the output is identical, latency usually improves, and for any workflow with a long, reused system prompt or a document that gets referenced across multiple turns, it's close to free money left on the table until you turn it on.

Way 2: Batch Anything That Doesn't Need an Answer Right Now

Not every AI call is a conversation. Nightly data enrichment, bulk classification, documentation generation, eval runs against a test set — none of that needs a response in the next two seconds. If it can wait a few hours, it shouldn't be paying real-time prices.

Both major providers now offer a flat, no-negotiation discount for exactly this case. OpenAI's own documentation describes its Batch API as processing a file of requests within a 24-hour window at a 50% cost discount compared to synchronous APIs — the same rate whether you're a five-person startup or a global enterprise, no volume tier to negotiate. Anthropic's Batch API offers a comparable structure, and its own pricing documentation notes explicitly that the batch discount stacks with prompt caching rather than replacing it.

Way 3: Stop Sending Every Query to Your Priciest Model

The default habit in a lot of AI deployments is to point every request at the flagship model, every time, regardless of whether the task actually needs it. Classifying a support ticket and drafting a legal summary are not the same job, and pricing them identically is how budgets run away.

The idea has a name — an "LLM cascade" — and it traces back to FrugalGPT, a 2023 Stanford paper by Lingjiao Chen, Matei Zaharia, and James Zou. The proposal: try a cheaper model first, and only escalate to a more expensive one when the cheaper model's answer looks unreliable. It's the paper nearly every routing tool since has built on or benchmarked against.

For evidence the concept holds up, a peer-reviewed 2025 follow-up from UC Berkeley, RouteLLM, published at ICLR, confirmed that a trained router can send most traffic to a cheaper model while preserving most of a stronger model's quality. For a concrete, real-world number, the best evidence I could find comes from a 2026 preprint that tested cascade routing on an actual production workload — 75,000 real labeled queries, not a lab benchmark — and measured 31% cost reduction at target accuracy. Worth flagging: that paper is a single-researcher preprint, not yet peer-reviewed, though its methodology (real production data, measured costs, confidence intervals, head-to-head comparison against baseline routing methods) is more rigorous than most of what passes for a "case study" in this space.

You don't need a research team to apply the idea. The practical version is a routing layer — increasingly built into AI gateways — that classifies incoming requests by complexity and sends the easy majority to a fast, inexpensive model while reserving the expensive one for genuinely hard cases. Unlike caching or batching, this one does carry a real tradeoff to manage: you're making a judgment call about which tasks can tolerate a cheaper model, so it's worth testing on a narrow workflow before rolling it out broadly.

Way 4: Get Visibility Before You Optimize Anything

Here's the uncomfortable part. Caching, batching, and routing all assume you already know where your AI spend is going — by team, by feature, by workflow. Most organizations don't yet, which is exactly why "AI cost management" topped the FinOps Foundation's skills-gap list instead of any specific optimization technique.

This is the part of the job that should feel familiar to anyone who's run a vendor evaluation: you don't approve spend you can't attribute, and you don't renew a line item nobody can explain. The same discipline applies here. Tag AI usage by team or feature from day one. Set budget alerts before you need them, not after finance asks why the bill tripled. And put AI spend on the same quarterly review cadence as any other significant vendor line — not because it's uniquely dangerous, but because it's new enough that nobody's built the habit yet.

---

The Framework: Do These in Order

Every technique above works. But the sequence matters more than any individual tactic, and skipping ahead is how teams end up optimizing the wrong thing. Here's the order that actually holds up:

1. Get visibility first. Tag AI spend by team and feature, and set alerting on anomalies before you touch anything else. You can't fix what you can't attribute.
2. Turn on caching and batching. Both are close to free wins — no quality tradeoff, often no code change beyond a config flag — so there's no reason to wait on these.
3. Introduce model routing deliberately. This is the one lever with a real tradeoff. Pilot it on a narrow, well-understood workflow, measure quality, then expand.
4. Review it quarterly, like any other vendor spend. AI pricing and model options are still moving fast enough that a setup that made sense six months ago may not be the best one today.

None of this requires a platform migration or a six-month initiative. It requires treating AI spend with the same basic financial discipline you'd apply to any other line item that grew this fast, this quickly — because at this point, that's exactly what it is.

Sources & further reading

• State of FinOps Survey: AI Value and Skills Top Priorities as FinOps Matures Across Technology Value FinOps Foundation via PR Newswire · Feb 2026
• FinOps teams gain clout as AI costs climb CIO Dive
• Prompt caching — Claude Platform Docs Anthropic (official documentation)
• Batch API guide OpenAI (official documentation)
• FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performance Chen, Zaharia & Zou — Stanford University, arXiv:2305.05176
• RouteLLM: Learning to Route LLMs with Preference Data Ong et al. — UC Berkeley / Anyscale / Canva, ICLR 2025, arXiv:2406.18665
• UCCI: Calibrated Uncertainty for Cost-Optimal LLM Cascade Routing Independent researcher, 2026 preprint (not peer-reviewed), arXiv:2605.18796 — production-workload results