What an AI feature actually costs, and how to forecast it
Tokens are the unit you get billed in, and almost nobody explains them in terms a budget holder can use. Here's the arithmetic, with real prices.
8 min readpublished and checked 2026-07-24
Every one of the manager-track AI certifications tests cost drivers, and for good reason: it's the question you'll actually get asked. Your CFO does not care about transformer architecture. They want to know what this costs at ten thousand users, and they usually want it before the next planning cycle.
The good news is that it's forecastable. Not the way a per-seat license is forecastable, but forecastable. Here's the arithmetic.
The unit is the token, and you pay both ways
A token is roughly three-quarters of a word in English. "Unbelievable" might be three tokens. "The" is one. The precision doesn't matter much for budgeting. What matters is that you are billed for the text going in as well as the text coming out, at different rates, and output costs several times more than input.
That asymmetry is where most estimates go wrong. People size the answer and forget they're also paying for the question. In a real system the question is usually far bigger than the answer, because it carries instructions, retrieved documents, and conversation history along with whatever the user actually typed.
Real numbers
Published prices as of July 2026, per million tokens, across one vendor's current lineup:
| Model | Input | Output |
|---|---|---|
| Frontier tier | $10 | $50 |
| Mid tier | $5 | $25 |
| Balanced tier | $3 | $15 |
| Fast tier | $1 | $5 |
The specific figures will move. Two things about the shape of the table won't.
Output costs five times input. Anything that makes the system chattier costs you disproportionately.
Top to bottom is a 10× spread. Model choice is the biggest single lever you have, and "just use the best one" is a decision with an invoice attached.
Working an example
Say a support-summarization feature. Every request carries:
- a 500-token instruction block
- a 2,000-token retrieved policy extract
- a 1,500-token customer conversation
and produces a 300-token summary. So 4,000 tokens in, 300 out.
On the balanced tier that's 4,000 × $3/1M = $0.012 of input, plus 300 × $15/1M = $0.0045 of output. About 1.65 cents per summary.
Ten thousand summaries a month comes to roughly $165. That's a perfectly fine answer to give a CFO, and it's nowhere near the number most people have braced themselves for.
Now change one variable. Move to the frontier tier and the same volume runs about $550. Still not frightening, but it's 3.3× for a quality difference you ought to be able to demonstrate before you commit to it.
The lever nobody mentions to managers
Go back to that example. The 500-token instruction block and most of the 2,000-token policy extract are identical on every single request. Only the customer conversation changes.
Every major platform now bills repeated content at a steep discount, provided the request is structured so the unchanging part comes first. Typically that's around a tenth of the normal input price on the repeated portion, against a small premium the first time it gets stored.
Apply it here. If 2,500 of the 4,000 input tokens are stable, you stop paying full price on 4,000 and start paying full price on 1,500 plus a tenth on 2,500. Input cost drops by more than half. Output quality doesn't change at all.
The reason this belongs to you and not just to your engineers is that it's an architecture decision, not an optimization someone does later. Build the system with the changing content at the front and the discount is simply unavailable until it's rebuilt. "Is the stable part of our prompt cached?" is probably the highest-leverage question a non-technical manager can ask in a design review.
Three ways the forecast goes wrong
Conversation history compounds. In a chat feature, every turn resends the whole conversation so far. Turn ten costs far more than turn one. If your estimate assumed a flat per-message cost, it's low, and sometimes badly low.
Retrieval size is a choice, not a constant. "Pull the twenty most relevant documents" costs four times what "pull the five most relevant" costs, on every request, forever. It's also frequently worse, because the irrelevant material dilutes the model's attention. Fewer and better-chosen usually wins on both cost and accuracy, which is a rare thing to be able to say about anything.
Retries don't show up in the demo. Systems that check their own work, or that retry when a response comes back badly formed, can quietly double or triple real cost against what a happy-path prototype suggested. Ask what the retry rate is. Nobody tends to volunteer it.
What to ask, and when
In the design review, before anything gets built:
- What's our estimated cost per transaction, and what's in the input?
- Which parts of the request are identical every time, and are we getting the repeat discount on them?
- What tier are we on, and how did the cheaper tier score on our own tests?
- Does cost scale with users, with usage, or with conversation length?
That last one determines how the line behaves as you grow, and it's the one that catches people out.
The framing that works with finance
Don't present AI cost as infrastructure. Present it as cost per unit of work (per ticket summarized, per document reviewed, per draft produced) and put it next to what that unit of work costs you today.
"$165 a month" invites a line-item argument. "1.65 cents a ticket against $4 of analyst time" is a conversation about margin, and margin is the conversation you want to be having.
Also worth reading
Get the next one.
One email when something new lands. Nothing else.