Why does Wavesteam advise most businesses against privately hosting Qwen or DeepSeek?

Wavesteam's default advice is that most private businesses should not host a foundation model themselves. Downloadable Qwen or DeepSeek weights do not deliver the quality, usable concurrency, reliability, and continuing upgrades of a production model service at a lower total cost. For knowledge Q&A, document work, content assistance, and business agents, a managed enterprise API usually leaves more budget for knowledge, permissions, workflow, integration, and evaluation.

This is our professional delivery judgement, not an industry statistic. “Open source,” “free weights,” and “data sovereignty” are not sufficient reasons to purchase servers. If a proof shows lower quality, inadequate capacity, or higher 36-month cost, we should recommend stopping the purchase.

When defining model, data, and production boundaries, also compare Should you use an LLM API or run your own models? A practical cost framework and Which companies can deliver a privately deployed AI assistant in China?; the linked guidance adds context that should be considered in the same decision.

Open weights are not an operated service

The enterprise still selects an inference engine and quantization, configures chat and tool calling, controls context and batching, builds safety, cache, monitoring, rollback, and upgrades, and validates every change. Qwen's official vLLM deployment guidance, rechecked on 26 August 2026, covers CUDA, parallelism, thinking mode, templates, quantization, and context configuration. The official DeepSeek-V3 repository describes several inference paths and large distributed examples. They prove deployability, not a ready enterprise SLA.

Smaller or aggressively quantized variants may fit hardware while losing the complex reasoning, long-document handling, structured output, or tool use the business needs. Any extra review, correction, and handoff becomes business cost.

Fixed compute rarely matches variable demand

APIs reduce cost during quiet periods; purchased capacity incurs depreciation, power/cooling, space/network, spares, and operations continuously. Compare:

36-month TCO = net hardware + facilities + software/support + implementation/migration + operations + upgrades/capacity + outage loss

Then divide all cost by tasks that simultaneously meet quality and response requirements. Failed attempts, human review, reserve capacity, and model upgrades remain in the numerator. If economics depend on sustained near-full utilization that the company does not have, the case fails.

“Runs locally” is not multi-user capacity. Input processing and token generation compete for memory and compute according to prompt/output length, thinking mode, and arrival pattern. Measure request and output-token throughput, time to first token, time per output token, and throughput that passes P95/P99 latency on representative tasks. vLLM's benchmarking tools separate these concepts. A vendor's “supports 50 users” without workload and latency is not procurement evidence.

Upgrades make hardware a continuing model-engineering program

Private operation owns model formats, drivers, inference frameworks, security updates, regression, and rolling releases. A new model may not fit existing devices or be supported promptly by the chosen stack. General server operations do not automatically supply model evaluation and inference optimization.

Multiple Mac minis can be useful for proofs, offline processing, or separate copies of a smaller model. Wavesteam does not recommend model-parallel Mac mini clusters as the default online production architecture. Apple's MLX LM and distributed MLX documentation show that distributed inference is possible; memory does not become one transparent shared pool, and generation can depend on cross-machine communication and the slowest node. If a model only fits by aggregating several machines, first test a smaller suitable model, managed API, hosted inference, and a standard GPU server under the same workload and TCO.

When private deployment deserves a proof

Proceed only when at least one hard condition exists:

• necessary content is contractually or operationally prohibited from entering any external model service;
• the system must run on an isolated or disconnected network;
• compliant managed services cannot meet proven local latency or continuity needs;
• a large, stable real workload and reusable GPU, facilities, and model operations already exist.

Even then, do not purchase first. The target model must pass an approved business evaluation, the target hardware must pass real concurrency and failure tests, and the 36-month budget must include redundancy, upgrades, and staff. Local operation still governs model files, update sources, logs, backups, and administrator access.

For most businesses, keep the application, knowledge, identity, business rules, audit, switching layer, and evaluation set under enterprise control while calling models as needed. Use a local small model for stable high-volume extraction or classification when it passes; reserve external models for harder tasks. If all content must remain inside, run a local PoC before procurement.

Wavesteam builds the task evaluation set, establishes an API or isolated baseline, derives peak and response targets from real logs, benchmarks candidates, and compares API, hosted, local small model, appliance, Mac mini, and GPU server over 36 months. The written decision is “continue API,” “use hybrid,” “enter private-deployment PoC,” or “stop procurement,” with scale, budget, evidence, and responsibility. Model, hardware, and cloud providers own their products and prices; Wavesteam owns the architecture, test method, integration, and contracted implementation.