How to write a knowledge base your AI agent won't hallucinate from
"The AI made something up" is the single most common complaint from new customers in the first 30 days. Almost every time, the cause isn't the AI — it's a vague, contradictory, or incomplete knowledge base. This is the practical guide to writing one that doesn't trigger hallucinations.
Why AIs hallucinate
When the AI is asked something its knowledge base doesn't cover, it has two choices: say "I don't know," or fill in a plausible-sounding answer based on general training. A well-configured AI is biased toward the first; a poorly-configured one falls back to the second.
What makes a knowledge base poorly configured:
- Vague phrasing ("we usually have appointments available next week") — the AI generalizes to fill the gap.
- Missing information on common topics — the AI improvises.
- Internal contradictions (the KB says hours are 9–5 but also "we close early on Fridays") — the AI picks one, often the wrong one.
- Implicit assumptions ("we take most major insurance") — the AI invents specifics.
The rule: facts only, no opinions, no hedging
Write the KB the way a contract reads. Specific, literal, no "usually" or "typically." Every sentence should answer a specific question with a specific answer.
| Bad | Good |
|---|---|
| "We usually accept most major insurance." | "We accept Delta Dental, Cigna, Aetna, and MetLife. We do not accept Humana or Medicaid." |
| "Same-day appointments are sometimes possible." | "Same-day appointments are available for existing patients with active emergencies. New patients are scheduled for the next available slot." |
| "Pricing varies by service." | "Cleaning + exam: $189. Whitening: $349. Crown: $1,200. Full price list at example.com/prices." |
| "We're typically open weekdays." | "Open Monday–Thursday 8am–5pm. Closed Friday, Saturday, Sunday. Holidays listed at example.com/hours." |
What to include
Eight categories cover 95% of inbound calls:
- Identity: exact business name, location, owner name, what the business does in one sentence.
- Hours: every day of the week, including holidays and closed days. Be specific.
- Services + prices (or "call for pricing" if pricing isn't published): every service the AI might be asked about.
- Booking flow: can the AI book directly, or does it capture intent and route? If booking: what slots are available, how far out, deposit required?
- FAQs: the top 10 questions callers ask, with exact answers.
- Insurance / payment methods: what's accepted, what isn't.
- Escalation triggers: the specific phrases that route to a human ("emergency," "lawsuit," "I want to speak to a manager," etc.).
- What to say when uncertain: the fallback phrase. "I don't have that information — I can take a message and have someone call you back. What's the best number to reach you?"
What to NOT include
- Anything you can't commit to literally. If the KB says "we'll call you back in 1 hour," the AI will promise that to every caller. You're now on the hook.
- Internal jargon. If you use codes or shorthand internally, translate to caller-facing language in the KB.
- "It depends" answers. If a question genuinely depends, write the answer as: "It depends on [X]. I'll capture your details and have someone call you back to confirm."
- Opinions about competitors. The AI doesn't disparage. It says: "I'm not in a position to compare with other providers. I can tell you what we offer."
- Aspirational claims. If you don't have 24/7 emergency coverage, the KB doesn't say you do. The AI shouldn't promise what the business can't deliver.
Test the KB before going live
Call the AI yourself. Try 10 questions:
- The 5 questions you put in the FAQ section (sanity check).
- 2 edge cases the FAQ doesn't cover (does the AI gracefully say "I don't have that info"?).
- 1 escalation trigger (does it route correctly?).
- 1 question about a service you don't offer (does it admit it doesn't, or invent something?).
- 1 deliberately ambiguous question (does it ask for clarification, or guess?).
If the AI invents an answer to any of these, the KB is incomplete. Add the missing info; test again. A clean test loop is the difference between a sticky customer and a 30-day churn.
Update cadence
Monthly review — check that hours, prices, and services are still accurate. Whenever a customer adds/removes services, raises prices, or changes hours — same day. The KB is the AI's reality; if it's out of date, the AI is out of date.
A simple convention: change-log at the top of the KB document. "2026-05-12: added new doctor's name to staff list. 2026-04-30: updated holiday hours." Five minutes of habit; saves arguments about when something changed.
The "what does the AI say when..." exercise
Once a quarter, sit with a notebook and write 20 questions a real caller might ask. For each, write down what the AI should say. Then test all 20 against the AI. The gaps are your KB updates for the quarter. This 60-minute exercise catches drift before customers do.