Does my AI agent need a model trained specifically for my business?
Wavesteam can provide fine-tuning, evaluation, and private model deployment, but most AI agents do not need a large model trained from scratch. We first establish a baseline with a capable foundation model, prompts, retrieval, tools, and a controlled workflow. Fine-tuning becomes justified only when repeatable tests show that model behavior remains the constraint and the gain outweighs the cost of data, training, and long-term operation.
Making a model understand a business does not necessarily mean changing its parameters. Current policies, products, equipment status, and orders keep changing; they belong in retrieval systems or business APIs where they can be updated, permissioned, and traced. Fine-tuning is better suited to stable behavior such as a strict output form, a classification boundary, a house style, or a recurring task strategy.
When defining model, data, and production boundaries, also compare What components does a production enterprise AI agent need?; the linked guidance adds context that should be considered in the same decision.
Four technical options
| Approach | Problem it solves | Data needed | Cost of change | Recommendation |
|---|---|---|---|---|
| Prompting and structured output | Defines the task, tone, fields, and rules | A small set of examples | Lowest; changes apply immediately | Start every project here |
| Retrieval and tool use | Supplies current knowledge, private data, and live business state | Documents, APIs, and permissions | Sources can change independently | Core architecture for most enterprise agents |
| Supervised fine-tuning or LoRA | Stabilizes format, style, classification, or domain-task behavior | Enough consistent, high-quality examples | Retraining, regression testing, and deployment | Use when the baseline repeatedly falls short |
| Pretraining from scratch | Creates a new foundation language or multimodal model | Massive corpora, compute, and a research team | Extremely high | Rarely appropriate for an ordinary enterprise application |
When fine-tuning can be worthwhile
It can make sense for a frequent, stable task such as high-volume classification, extraction, or structured generation, particularly when a tuned smaller model can reduce latency and unit cost. It may also help when schemas and examples still fail to produce a sufficiently consistent format, when the organization owns lawful and well-labelled examples of a stable specialist task, or when an offline open model must be improved for known workloads on fixed hardware.
Training is not the answer when the model does not know yesterday's price; connect the database. If it cannot find a policy, improve the source documents and retrieval. If it calls a refund tool incorrectly, fix permissions, workflow, and parameter validation. Fine-tuning does not inherently eliminate hallucination, excessive privilege, or prompt injection.
Establish a baseline before training
Build a representative evaluation set—often at least 100 samples, stratified by task and risk—and record task success, important failure types, P95 latency, and unit cost without tuning. Test better instructions, examples, retrieval, tools, and suitable foundation models in controlled steps. Only a persistent error that survives these changes should become a tuning target.
Separate training, validation, and final test material by source or time so near-duplicate templates do not leak across sets. For every training item, establish provenance, usage rights, personal-data handling, and quality. Inconsistent labels and weak synthetic answers teach the defect rather than correcting it. After tuning, regress not only the target task but also general capability, safety refusals, and tool parameters; a local improvement must not hide a broader decline.
The deliverable is more than a weight file
A usable model handover identifies the foundation model and licence, data inventory and processing record, training configuration and versions, evaluation sets and results, inference image, hardware capacity, monitoring, rollback, and upgrade plan. A commercial fine-tuning platform also requires review of data terms, model lifecycle, export options, and provider retirement policy. Ownership of weights, training data, code, and derivative work must be explicit.
Wavesteam begins with a “should we train?” review. If retrieval and integration solve the requirement, we deliver the more maintainable agent. When evaluation supports tuning, we can provide data preparation, LoRA or platform fine-tuning, private inference, and regression testing. Our QuanYuTong case study demonstrates experience with an AI content product; it does not, by itself, prove that a new client's task needs training.
References
- OpenAI's model optimization guide describes the iterative relationship among evaluation, prompting, and fine-tuning; available models can change.
- The Hugging Face PEFT documentation covers parameter-efficient methods such as LoRA.
- The original Retrieval-Augmented Generation paper explains how external retrieval supplies knowledge without encoding every fact in model parameters.
The decision to train should rest on measured improvement on the same test set, total cost, data rights, and a named owner for continuing maintenance.