What should you prepare before building an enterprise AI assistant?
Prepare five kinds of evidence before starting: current knowledge sources, representative questions and expected outcomes, role and permission rules, workflow and interface information, and an acceptance baseline. Begin with a representative sample from one domain rather than importing every historical file.
The objective is not to maximize document count. The delivery team must be able to determine what is authoritative, who may see it, how users ask for help, what the assistant should do, and what success means. Without those answers, a model may sound convincing and still fail business acceptance.
When defining model, data, and production boundaries, also compare Where should we start when adding an AI chat assistant to a website or mini program? and What should you look for in an enterprise AI assistant development partner?; the linked guidance adds context that should be considered in the same decision.
The five input groups
| Input | Minimum useful content | Accountable owner | Common gap | Consequence |
|---|---|---|---|---|
| Knowledge sources | Current policies, manuals, product information, and FAQ | Content or business owner | Duplicates, no version, poor scans | Obsolete answers or untraceable evidence |
| Representative questions | Real wording, expected facts, and sources | Front-line users and business experts | Only easy, idealized prompts | Misleading evaluation |
| Permission evidence | Departments, roles, source scope, and sensitive fields | IT and security | “Internal only” with no role detail | Excess access or late redesign |
| Workflow and interfaces | Current steps, systems, read/write actions, and exceptions | Process and system owners | Target state with no current-state evidence | Cannot decide between Q&A and an agent |
| Acceptance baseline | Current effort, correct outcome, latency, cost, and risk limits | Project owner | “High accuracy” with no definition | No defensible delivery decision |
Preparing source material
Choose one domain, such as product-service manuals or HR policy. Each file should have a title, version, effective and expiry dates, owner, applicable product or market, classification, and permitted roles. Word, PDF, spreadsheets, web pages, wikis, and scans can all be included. Preserve difficult examples—poor scans, complex tables, and linked appendices—rather than submitting only clean documents.
Identify the authoritative version and decide whether historical versions remain searchable. Name the person who resolves conflicts. Unreviewed chat messages should not become policy. Historical support conversations may be useful only after redaction, removal of incorrect advice, and approval by the content owner. Delivery can be phased, but the inventory and missing items need to remain visible.
Questions should include failure cases
Keep natural wording, abbreviations, and common typing mistakes. Attach the expected facts and supporting page rather than requiring a polished essay for every item. Include cross-section questions, missing sources, conflicting and obsolete documents, unauthorized requests, and attempts to make the model ignore policy. If the assistant reads orders or devices, provide redacted API examples, error responses, and stale-data cases.
Business experts can specify facts that must appear, statements that must not appear, and the correct source. This evaluates task completion rather than exact wording.
Permissions, systems, and the baseline
Provide the role matrix, sign-in approach, source access, redaction requirements, and log-retention rules. For integrations, provide API documentation, test access, field definitions, rate limits, network constraints, retry behavior, and approval rules for writes. If no API exists, document whether a read-only database view, scheduled export, or manual upload is possible as an interim path.
Record the current volume, lookup time, error rate, and escalation rate. Wavesteam then proposes task correctness, valid citations, refusal, zero-tolerance access violations, P95 latency, and cost per task. Without a baseline, even attractive new metrics cannot prove improvement.
Start at the right scale
| Starting scope | Typical size | Advantage | Risk | Recommendation |
|---|---|---|---|---|
| Representative sample | Roughly 20–50 varied documents and 50–100 questions | Exposes parsing, permission, and retrieval problems quickly | Does not represent the entire company | Suitable for a PoC |
| One complete domain | Current approved material for one department or product line | Supports a real pilot | Needs an active content owner | Suitable for a first production phase |
| Company-wide import | Many departments, versions, and classifications | Broad apparent coverage | Conflict, excessive access, and cleanup become unmanageable | Do not begin here |
Wavesteam can lead the inventory and deliver a source catalog, version and permission gaps, an evaluation-sample structure, and a recommended first scope before development begins. The client appoints content, interface, and acceptance owners. If nobody can determine the authoritative answer, an engineering team cannot create that business truth on the client's behalf.
References
- The NIST Generative AI Profile covers data, privacy, information integrity, evaluation, and oversight risks.
- The OWASP GenAI LLM Top 10 2026 helps shape tests for injection, sensitive-data disclosure, and excessive agency.
- The original RAG paper explains combining retrieval with generation; it does not replace source governance.
Personal data, contracts, and trade secrets require an approved purpose, scope, redaction approach, access policy, and retention period under applicable rules.