How to Stop a Support Chatbot From Making Things Up

Every support chatbot demo looks the same. Someone asks "what are your business hours?", the bot answers correctly, everyone nods. Then it goes live, a customer asks whether the enterprise plan supports SSO — something the docs never mention — and the bot says yes. Confidently. With a setup guide it invented on the spot.

That one answer costs more than the bot ever saved on support tickets. The customer signs, discovers the feature doesn't exist, and now a human is handling an escalation the bot created. This failure mode has a boring, structural cause, and a boring, structural fix. Neither involves buying a smarter model.

Why chatbots make things up

A language model's default behaviour is to answer. It was trained to continue text plausibly, and a fluent guess and a retrieved fact look identical in the chat window. Most chatbot builds wire the model straight to the user with the company docs stuffed loosely into the prompt, which works right up until one of three things happens:

• Retrieval misses. The right passage exists, but the search over your knowledge base didn't surface it — usually because the docs were chunked at arbitrary character counts and the answer got split across two fragments, neither of which scored high enough to be retrieved.
• The document never indexed. Uploading a file and having it available for retrieval are two different events. A document that failed indexing is invisible to the bot, and nothing in the chat interface tells you.
• The question is genuinely outside the docs. No help centre covers everything. Without an explicit instruction about what to do here, the model fills the gap — fluently.

Notice that none of these are model problems. They are pipeline problems, and they respond to pipeline fixes.

Grounding: the structural fix

A grounded bot is one that is only allowed to answer from a knowledge base it retrieves against, and that has a defined behaviour for the case where retrieval comes back empty. In practice that means four settings working together:

The score threshold is the one most builds skip, and it is the difference between "no strong match found, escalating" and "here's a weak match dressed up as an answer". The refusal instruction is the other half of the same contract: the bot needs a line it is required to say when the knowledge base doesn't cover the question — something like "I don't have that in my knowledge base — I'll flag this for a human." An escalation is a good outcome. It is the bot correctly recognising the edge of what it knows.

Citations are for you, not the customer

Switch citations on even if you never show them to end users. When a customer disputes an answer, you want to see exactly which document and which chunk the bot answered from. Without citations, every complaint becomes an archaeology project. With them, it's a one-minute check: either the doc is wrong (fix the doc) or retrieval surfaced the wrong passage (fix the chunking).

The knowledge base decides everything upstream

Retrieval quality is mostly determined before the first question is ever asked, at ingestion time. Two rules do most of the work:

Chunk on headings, not character counts. A help-centre article has natural units — a section under a heading answers one question. Split there and retrieval returns whole answers. Split every 500 characters and the pricing table gets separated from the sentence explaining it, and neither half ranks well for a pricing question.

Verify indexing, continuously. Treat "is every document indexed?" as a monitoring question, not a setup question. A re-ingest that silently fails leaves the bot answering from last month's policy. A status check with a clean exit code — 0 when everything is indexed, 1 when it isn't — can sit in cron and email you when the bot's knowledge quietly degrades.

It also matters what you feed in. Marketing pages poison a support knowledge base — they are written to persuade, and the bot will happily retrieve "the fastest solution on the market" as if it were a fact. Ingest the help centre, the policy pages and the changelog. Leave the landing pages out.

Test like a customer who wants the bot to fail

Before the bot meets a real customer, spend thirty minutes asking it three kinds of questions:

• Covered questions — things the docs answer. The bot should answer them, with the right citation.
• Near-miss questions — plausible questions the docs almost answer. This is where ungrounded bots start improvising, and where a good score threshold shows its value.
• Out-of-scope questions — refund edge cases, legal questions, feature promises. The only acceptable behaviour is the refusal line and an escalation.

If the bot invents anything during the near-miss round, the fix is almost always in the knowledge base or the threshold, not in the prompt. Re-chunk, re-test, and only then go live.

Do it yourself, or take the shortcut

Everything above is buildable by hand on any RAG stack — Dify, Flowise, LangChain, or your own retrieval pipeline. If you'd rather start from a working implementation, our Chatbot Knowledge-Base Starter packages it for Dify: four dependency-free Python scripts (create the KB, ingest a folder with heading-aware chunking, crawl your own sitemap, verify indexing from cron), an importable app config with the grounding prompt, citations and refusal line already wired, and the playbook with the 30-minute test protocol. It was live-tested end to end against a real Dify instance before we listed it.

And if you'd rather not run any of it yourself, we build and host grounded support bots as a service — same rules, our infrastructure.

Either way, the principle is the one your customers already hold you to: a support answer is either true and traceable to a document, or it's an escalation. There is no third category worth shipping.