Should you use an LLM API or run your own models? A practical cost framework

Our default recommendation is to start with a model API. Once you provide representative tasks, data constraints, budget boundaries, and the business outcomes that matter, Wavesteam can use pilot logs to model quality, peak demand, and three-year total cost of ownership (TCO). Self-hosting makes sense only when data or network requirements leave no alternative, or when sustained utilization produces a genuine cost advantage. Many established workloads ultimately benefit from a hybrid approach: APIs for complex tasks and smaller models for routine ones.

The headline API price does not tell you what an AI application will cost. Your bill depends on task volume, input and output tokens per task, peak concurrency, long-context use, retries, retrieval and reranking, observability, and human review.

When defining model, data, and production boundaries, also compare Why does Wavesteam advise most businesses against privately hosting Qwen or DeepSeek? and Why does an application using a large-model API still answer inaccurately or invent facts?; the linked guidance adds context that should be considered in the same decision.

Three deployment paths

Path	Best stage or situation	Main cost drivers	Advantages	Key risk and recommendation
Model API	Proofs of concept, variable demand, or access to current frontier models	Input/output tokens, cached input, embeddings, and tool calls	No GPU operations; models can be changed quickly	Best starting point, with usage and budget controls from day one
API + locally hosted small models	Stable volume with tasks that can be separated by difficulty	API usage, local inference, routing, and evaluation	A useful balance of quality, cost, and elasticity	The first scale-up option for many business applications
Fully self-hosted or private deployment	Data cannot leave a controlled environment, offline operation is mandatory, or demand is consistently high	GPUs, cloud or data-center capacity, inference software, monitoring, upgrades, and engineering staff	Maximum control over data and model versions	Buying a GPU is not an operating model; avoid this route without a capable platform team

Turning business demand into a defensible budget

Wavesteam breaks representative usage into tasks, inputs, outputs, cacheable content, tool calls, and retries. A basic API estimate is:

Monthly model cost = monthly tasks × (average input tokens × input rate + average output tokens × output rate) + other model calls

“Other model calls” may include embeddings, reranking, speech, image processing, moderation, and retries. For retrieval-augmented Q&A, retrieved passages are part of the input. For an agent, one user request may trigger several planning rounds and tool calls. Estimating only the final answer shown to the user will understate the cost.

A self-hosted estimate must include inference capacity, resilience headroom, storage and networking, deployment and monitoring, model upgrades, quantization work, security patches, and engineering time. We compare these costs across 36 months and state assumptions such as residual hardware value, peak capacity, and staffing separately. The client does not need to estimate tokens, select a quantization method, or translate concurrency into GPU counts. You confirm the business-volume assumptions and budget constraints; Wavesteam owns the technical model.

Optimize in the right order

Remove waste before shopping for a cheaper model. A sensible sequence is:

1. shorten unnecessary system instructions and retrieved context;
2. use provider-side caching for repeated prefixes;
3. route classification and extraction tasks to smaller models;
4. add expiring semantic caches where repeated questions are genuinely equivalent;
5. cap agent steps and tool retries;
6. batch non-interactive work;
7. only then compare providers or self-hosting.

Every cost change must be tested against quality. Wavesteam builds a fixed evaluation set around business outcomes approved by the client, then tracks task success, human takeover, P95 latency, cost per successful task, and monthly peaks. A lower token count is not a saving if more tasks fail or require manual correction.

A decision gate you can actually use

We normally recommend two to four weeks of real or shadow traffic. From that run, we capture average tokens, peak demand, cache hit rate, retry rate, and human-review volume. The output is a comparable 36-month TCO for API, hybrid, and private deployment, plus a recommended path, the usage threshold that would change the recommendation, and exit conditions.

If self-hosting appears cheaper only at the raw inference level but loses its advantage after resilience, staffing, and upgrades are included, adding that complexity in the name of “control” is not a sound business decision.

Reference points

• OpenAI model and pricing comparison illustrates one mainstream API's public pricing units for input, cached input, and output tokens. A real procurement exercise should compare more than one provider.
• Google Cloud Pricing Calculator can estimate cloud infrastructure but does not include internal engineering and management costs.
• The FinOps FOCUS Specification provides a common structure for cloud-cost data across services and providers.

Model capabilities and prices change frequently. Recheck official pricing, contractual terms, and measured production-like traffic at the time of implementation.