The question that decides whether an AI feature survives contact with a finance team is not "what does a million tokens cost." It is "what does one completed task cost, including the times it fails."
Those two numbers are usually an order of magnitude apart. Understanding why is most of the work.
Per-call pricing versus per-task cost
A single model call is cheap and predictable. A task is neither, because a task is a loop. The agent retrieves context, calls a tool, evaluates the result, and often does that several times before producing something acceptable. Each pass carries the accumulated conversation forward.
The drivers that actually move the number:
| Driver | Why it multiplies cost |
|---|---|
| Turns per task | Each turn re-sends accumulated context. Cost grows faster than linearly. |
| Retrieved context size | Retrieval is usually the largest share of input tokens, not the user's question. |
| Retry and repair rate | Failed runs cost the same as successful ones. A 20% failure rate is a 25% cost increase. |
| Tool-call verbosity | Raw API responses fed back verbatim can dwarf everything else in the context. |
| Guardrail and evaluation passes | Every safety or quality check is another call, often on a comparable amount of context. |
None of these appear in a demo, because a demo is one clean pass on a friendly input.
Estimating before you build
You do not need a working system to get within range. You need a representative sample of real inputs - twenty is often enough to be directionally right - and an honest estimate of four things: average turns to completion, average context carried per turn, the share of runs that need a retry, and how many verification passes you intend to run.
Multiply it out, then double it. The doubling is not pessimism; it accounts for the edge cases in your real input distribution that your twenty samples did not contain.
Compare the result to what the task costs today when a person does it. If the ratio is not comfortable, the answer is not a better prompt. It is a narrower task.
Where the cost actually goes down
In roughly the order we find them useful:
- Narrow the task. The single largest lever, and the one teams reach for last. An agent that handles one well-defined job in three turns beats a general assistant that averages nine.
- Control what enters the context. Summarise tool outputs before feeding them back. Retrieve fewer, better passages rather than more, hopeful ones. Most systems are carrying context that changes no decision.
- Route by difficulty. Most tasks in a real distribution are easy. Sending all of them to your most capable model is paying premium rates for work a smaller one handles identically.
- Cache the stable prefix. System instructions, schemas and reference material that do not change between calls should not be re-billed as fresh input on every turn.
- Fail faster. A run that will not succeed should stop at turn three, not turn twelve. This requires knowing what failure looks like, which requires evaluation.
That last point is the one that ties everything together.
Evaluation is a cost control, not a quality nicety
Without a test set you cannot tell whether a cheaper configuration is also a worse one. Every optimisation above becomes a guess, and teams either leave money on the table because they are afraid to change anything, or ship a regression they discover through complaints.
A usable harness is smaller than people expect: a few dozen representative tasks, an expected outcome or an acceptable-range definition for each, and a way to run the whole set on demand. Once that exists, cost and quality become a curve you can move along deliberately instead of a trade-off you argue about.
It is also the artefact that makes the business case defensible. "Cost per task fell 40% with no measurable change in accuracy across our 60-case suite" is a sentence a finance team can act on. "It feels about as good and seems cheaper" is not.
The number worth putting in the business case
Cost per successfully completed task, measured on real inputs, including failures and verification, with the accuracy figure it was achieved at.
One number, one accuracy figure, both reproducible. Everything else - tokens, calls, model names - is implementation detail that will change three times before the system reaches production anyway.