A cost model is a deliverable

A language model pilot costs almost nothing. A thousand requests a day at a few rupees each is a rounding error. The same system at sixty thousand requests a day with retries, a retrieval step and a verification pass is a line item the finance director will want to discuss, and by then the architecture is fixed.

We treat the cost model as a deliverable with the same status as the code. It is published with the baseline in week four, revised every fortnight, and reviewed in the same pull request as the change that affects it.

What the model contains

Per request: the tokens in, the tokens out, the number of model calls including retries, the retrieval cost, and the infrastructure share. Per month: the request volume by hour, because the volume is not flat and the provider’s rate limits are not either. Per decision: what a threshold change does to the number of requests that escalate to a human, because the human is the most expensive component in the system and the model’s job is to use them well.

Why it changes the architecture

Take a triage system as a worked example. The cost per item rarely falls because of a cheaper model. It falls because the retrieval step gets cached, because simple cases are routed to a smaller classifier, and because a retry loop that the demo never exercised is found and fixed. Every one of those changes gets made because the cost model shows where the money goes. Without it, the team argues about model choice, which is usually the one variable that does not matter.

Where it lives

In your repository, next to the evals, as a spreadsheet and as telemetry. The telemetry is the part that keeps it honest: the spreadsheet says what we expected, the per-request logs say what happened, and the fortnightly review is where the two are reconciled. When we leave, the platform lead owns the review, and is free to change it. That is the point.