FinOps for AI: Creating a Playbook for Managing AI Spend

TL;DR: AI spend is exploding, and most of it comes from API calls made by agents and applications — not from per-user chat subscriptions. A FinOps for AI practice measures results per dollar spent across models, apps, and agents, then routes each workload to the cheapest model that meets the quality bar. As of July 2026, US open-weight models deliver roughly 112.5 intelligence-index points per blended dollar versus 7.6 for US frontier models — about a 15x difference — which is why an AI spend playbook matters.

What is FinOps for AI?

FinOps, as defined by the FinOps Foundation, is an operational framework and cultural practice that maximizes the business value of technology through timely, data-driven decisions and shared financial accountability across engineering, finance, and business teams. FinOps for AI applies that discipline to AI spending: optimizing outcomes across technical capabilities, people, and spend, with engineering, finance, product, and ITAM all at the table.

AI terminology you need for cost management

AI models. An AI model is a program that can autonomously make decisions or predictions. For cost purposes, they fall into two camps:

• Closed-weight (frontier) models — e.g., Anthropic Claude, OpenAI ChatGPT, Microsoft Copilot, Google Gemini. These are the highest-performing and most costly models, hosted only by the AI vendor and accessed via API.
• Open-weight models — e.g., DeepSeek, Alibaba Qwen, Meta Llama, Google Gemma, Microsoft Phi, Mistral AI. Model parameters are released with the model (not fully open-sourced). They perform lower than frontier models but cost far less, and can be vendor-hosted or self-hosted.

Dimension

Open-weight

Closed (proprietary)

Access to weights

Yes — downloadable

No — API only

Self-hosting

Possible

No

Fine-tuning

Yes, with the right hardware

Limited or none

Data privacy

Full control

Provider dependent

Cost at scale

Lower (hardware cost)

API pricing per token

Latest capabilities

Usually behind

At the frontier

Tokens. Tokens are the unit AI vendors bill in — units only the model really "knows," but they are what you pay for. Input tokens are prompts from users, apps, or agents; output tokens are the model's results.

How are AI models priced?

There are two pricing models, and only one of them is where budgets break:

Per user — flat monthly seat pricing for chat interfaces, with input restricted to actual humans (text, images, voice) and limits on requests. This is not the high-spend area.

API calls by agents or applications — priced per input and output token. This is the high-spend area, because agents and applications can generate far more requests than people ever will.

Per-user plan pricing (published, as of July 2026)

Provider

Plan tier

Price ($/user/month)

Claude (Anthropic)

Team / Enterprise (seat)

$20

Microsoft 365 Copilot

Business (SMB, ≤300 users)

$18

Microsoft 365 Copilot

Enterprise

$30

ChatGPT (OpenAI)

Business (formerly Team)

$20

ChatGPT (OpenAI)

Enterprise

$55

Gemini (Google Workspace)

Business Starter

$7

Gemini (Google Workspace)

Business Standard

$14

Gemini (Google Workspace)

Business Plus

$22

Gemini (Google Workspace)

AI Expanded Access (add-on)

$20

API token pricing — US frontier vendors (as of July 2026)

Vendor

Model

Tier

Input ($/MTok)

Output ($/MTok)

Cached input ($/MTok)

Anthropic (Claude)

Opus 4.8

Flagship

$5.00

$25.00

$0.50

Anthropic (Claude)

Fable 5

Flagship (creative)

$10.00

$50.00

—

Anthropic (Claude)

Sonnet 5

Mid

$2.00

$10.00

—

Anthropic (Claude)

Haiku 4.5

Budget

$1.00

$5.00

—

OpenAI (ChatGPT API)

GPT-5.6 Sol

Flagship

$5.00

$30.00

$0.50

OpenAI (ChatGPT API)

GPT-5.6 Terra

Mid

$2.50

$15.00

$0.25

OpenAI (ChatGPT API)

GPT-5.6 Luna

Budget

$1.00

$6.00

—

OpenAI (ChatGPT API)

GPT-5.4 Nano

Ultra-budget

$0.20

$1.25

—

Google (Gemini API)

Gemini 3.1 Pro Preview

Flagship

$2.00

$12.00

—

Google (Gemini API)

Gemini 3.5 Flash

Mid

$1.50

$9.00

—

Google (Gemini API)

Gemini 2.5 Flash-Lite

Budget

$0.10

$0.40

—

Average API token price by model group (as of July 2026)

Model group

Avg input ($/1M tokens)

Avg output ($/1M tokens)

US frontier & mini models (n=8)

$1.82

$10.51

Chinese models (n=13)

$0.69

$2.90

US open-weight, excl. Mistral (n=11)

$0.13

$0.51

Chinese vendors (DeepSeek, Alibaba Qwen, Moonshot Kimi, Zhipu GLM, MiniMax, Baidu Ernie, ByteDance Doubao) and US/French open-weight families (Meta Llama, Google Gemma, Microsoft Phi, NVIDIA Nemotron, OpenAI gpt-oss, Mistral) price dramatically below US frontier tiers — open-weight input tokens average roughly 14x cheaper and output tokens roughly 20x cheaper than US frontier models.

When does self-hosting an open-weight model beat the API?

Self-hosting only pays off above a breakeven token volume — the monthly volume at which a rented GPU cluster costs the same as the model's API rate. Below that volume, the API is cheaper. Belarc's engineering estimates (H100 at ~$2.50/hr, 24/7 utilization, 50M/10M input-output mix, July 2026), which exclude engineering/ops overhead:

Model

Breakeven volume

Cluster assumption

API price (in/out per MTok)

NVIDIA Nemotron 3 Ultra (550B total / 55B active MoE)

8.6B tokens/month

4x H100 80GB · $7,200/mo

$0.50 / $2.50

Meta Llama 4 Maverick (400B total / 17B active MoE)

32.0B tokens/month

4x H100 80GB · $7,200/mo

$0.15 / $0.60

gpt-oss-120b (~120B total, small MoE active)

63.5B tokens/month

2x H100 80GB · $3,600/mo

$0.04 / $0.14

Real-world breakeven volumes run somewhat higher once engineering and operations overhead are included. The takeaway: unless a workload consistently pushes billions of tokens per month, vendor-hosted APIs usually win on cost.

What should you measure?

The ideal metric is your results per dollar spent — for each AI model, app, agent, and hosting choice (SaaS or self-hosted). Since true business results are hard to instrument at first, use these proxies:

• Token prices (input and output, per model)
• Results from benchmarks
• Performance = benchmark results ÷ token prices

How to measure results (benchmarks)

• Artificial Analysis Intelligence Index — a composite benchmark of AI capabilities spanning math, science, coding, and reasoning.
• Epoch Capabilities Index — a composite of 50 AI benchmarks, with open-source methodology and documentation. Epoch's data shows open-weight models trail the frontier by only about four months.
• Industry-specific benchmarks — legal, healthcare, finance, customer service, software development, and general agents.

Intelligence Index by model group (July 2026, index v4.1)

Model group

Average score

Notable models

US frontier

53.7

GPT-5.6 Sol (59), Claude Opus 4.8 (56), Gemini 3.1 Pro (46)

Chinese frontier

54.9

Kimi K3 (57), Qwen3.7 Max (56.6), GLM-5.2 (51)

US open-weight

39.0

Nemotron 3 Ultra (48), Gemma 4 31B (39), gpt-oss-120b (33)

Scores vary by source and index version, so treat them as directional rather than exact.

Performance per dollar: the number that changes decisions

Dividing average Intelligence Index score by average blended token price (50M input + 10M output mix, July 2026):

Model group

Performance per dollar

Basis

US open-weight

112.5 index pts / $

Index 39.0 @ $0.35/MTok blended

Chinese frontier

19.2 index pts / $

Index 54.9 @ $2.86/MTok blended

US frontier

7.6 index pts / $

Index 53.7 @ $7.06/MTok blended

US open-weight models deliver roughly 15x more measured intelligence per dollar than US frontier models. Frontier models still win where maximum capability is required — but a large share of enterprise AI workloads don't require it.

Your AI spend playbook: 4 steps

Focus on your high-cost apps and agents. API usage — not per-user seats — is where spend concentrates. Find it first.

Ask whether those apps can run on open-weight or mini models. Evaluate US, French, and Chinese options, SaaS-hosted or self-hosted, against your quality bar.

Get accurate, up-to-date data on costs and usage. Real-time, attributed to specific agents and applications, and taking full advantage of cache and batch pricing savings.

Continuously update your playbook. Everything in AI — prices, models, benchmarks — is changing fast. A playbook written six months ago is already stale.

Frequently asked questions

What is FinOps for AI?

FinOps for AI applies the FinOps discipline — cross-team financial accountability and data-driven technology decisions — to AI spending. It means measuring results per dollar across models, apps, and agents, and routing each workload to the most cost-effective model that meets requirements.

Where does most enterprise AI spend come from?

From API calls made by agents and applications, priced per input and output token. Per-user chat subscriptions ($7–$55/user/month) are comparatively small because humans are rate-limited; agents are not.

Are open-weight models good enough to replace frontier models?

For many workloads, yes. Open-weight models trail the frontier by only about four months on the Epoch Capabilities Index, and they deliver roughly 15x more intelligence per dollar than US frontier models as of July 2026. Frontier models remain the right choice where maximum capability matters.

When is self-hosting cheaper than using an API?

Only at very high volume. Breakeven ranges from roughly 8.6B tokens/month (Nemotron 3 Ultra) to 63.5B tokens/month (gpt-oss-120b) on rented H100 clusters — and real-world breakevens are higher once engineering overhead is included.

How should I measure AI model value?

Use results per dollar. As proxies, combine token prices with composite benchmarks like the Artificial Analysis Intelligence Index or the Epoch Capabilities Index, plus industry-specific benchmarks for legal, healthcare, finance, customer service, software development, and agents.

How do cached and batch tokens reduce AI costs?

Vendors discount cached input tokens steeply — for example, $0.50/MTok cached vs. $5.00/MTok standard input on flagship tiers — so structuring apps to reuse cached context and batch requests directly cuts spend.

Belarc, Inc. helps organizations get accurate, real-time data on software and AI costs and usage across their entire enterprise. Belarc serves over 1,800 customers in more than 50 countries and holds eight US and worldwide patents. Contact Sumin Tchen at stchen@belarc.com, info@belarc.com, or +1 978-461-1100.