Explainer

RAG chatbot explained without the jargon

Every chatbot vendor now says their bot uses RAG. Almost none explain what it means. It is the difference between a bot that answers from your business and one that answers from the internet's general impression of businesses like yours.

By Casper LöfgrenPublished Jul 20, 20267 min readSource: Lewis et al. 2020, Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (arXiv 2005.11401).

An open recipe book with a hand pointing at a line

RAG stands for retrieval-augmented generation. Three words, one of which is doing all the work.

Here is the whole idea in one sentence: instead of asking the language model what it knows, you look up the relevant facts in your own content first, then ask the model to answer using only those.

That is genuinely it. Everything else is implementation detail.

The term comes from a 2020 research paper: Lewis et al., Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks.

Why a RAG chatbot exists at all

A language model has read an enormous amount of text and absorbed the patterns in it. What it has never read is your delivery terms, your return window or your product range. It has no idea you stopped stocking the blue one in March.

Ask a model directly about your return policy and it will produce something. Fluent, plausible, formatted like a return policy, and invented. That is what people mean by hallucination: the model doing exactly what it was asked, with nothing true to work from.

RAG changes the question. Instead of "what is this company's return policy?", the model gets three passages from your own site and one instruction: answer the customer using only these. That is a task it is genuinely good at.

What actually happens when a customer types a question

Four steps, every time someone asks:

  1. The question is turned into a search. Not a keyword search, a meaning-based one. A customer asking for "something for a wet room" can match a page that only ever says "bathroom".
  2. The most relevant passages are retrieved from your indexed content: product pages, FAQs, delivery terms, whatever you gave it.
  3. Those passages go to the model with the question and with instructions about tone, scope and what to do when the answer is not there.
  4. The model writes the answer from that material, in your voice, at the length you asked for.

The consequence is the part that matters. If the answer is not in your content, a well-built RAG system says so instead of inventing one, because it was told to use the passages provided and there were none.

What RAG does not fix

Vendors present RAG as a cure for hallucination. It is a large reduction, not an elimination. The three failure modes below are worth knowing, because they tell you exactly what to test.

Retrieval can fetch the wrong passage

If the search step returns a page about the wrong product, the model writes a confident answer from the wrong source. The text is grounded, just in the wrong ground. This is the most common failure in production, and it looks exactly like a correct answer.

Your content can be wrong or stale

RAG faithfully reproduces whatever you gave it. A price that changed six months ago, on a page nobody updated, becomes the bot's confident answer. The bot is not wrong; the website is, and now it has a voice.

Combining two passages can produce a false conclusion

Say one page promises free delivery on orders over 50 €, and another promises free delivery to Canada. A model can merge the two into a claim neither page makes: free delivery to Canada on every order. Good systems are instructed conservatively, and no system is immune.

The four questions worth asking a vendor

"Do you use RAG?" is a question every vendor now answers yes to, which makes it useless. These four are not.

QuestionWhat a good answer sounds like
What does it do when the answer is not in my content? It says it does not know and offers a route to a human. If the answer is "it does its best", that is hallucination with a friendlier name.
Can I see which source it used for a given answer? Yes, per conversation. Without that you cannot debug a wrong answer, you can only guess at it.
How often does it re-read my site? A stated schedule, plus a manual re-index on demand. "At setup" means your bot quietly goes out of date.
What happens when I fix a wrong answer? The correction applies to the next conversation, and you can see it in the dashboard. If corrections need a support ticket, expect never to make them.

RAG versus fine-tuning

The other approach you will hear about is fine-tuning: training a model further on your material, so the knowledge is baked into the model itself.

For a customer-facing business chatbot, RAG wins on nearly every practical axis. Here is why, so you do not have to take it on faith:

  • Updating. RAG updates when your website updates. Fine-tuning needs a retraining run.
  • Traceability. RAG can point at the passage it used. A fine-tuned model cannot say where an answer came from.
  • Cost. Re-indexing a site is cheap and routine. Retraining is neither.
  • Correction. With RAG you fix the source and the answer changes. With fine-tuning, the unwanted behavior sits inside the model.

Fine-tuning still has a place, usually for tone and format rather than facts. Most production systems that look like one or the other are quietly both: retrieval for what is true, light tuning for how it sounds.

What this means when you are buying

RAG is why a modern chatbot no longer needs you to write question-and-answer pairs by hand. The older generation of tools asked you to anticipate every question. A retrieval-based bot reads what you already published and works out the rest, so setup collapsed from weeks to an afternoon.

It also moves the work. You are no longer writing bot answers, you are making sure your website says what you want said. Those are the same job, and only one of them was ever worth doing.

The practical test: ask a prospective bot something your website genuinely does not cover. A good one tells you it does not know, a bad one writes you a paragraph. The test takes thirty seconds and tells you more than any feature list.

Frequently asked

What does RAG mean in a chatbot?+

RAG stands for retrieval-augmented generation. Instead of relying on what a language model already knows, the system first retrieves relevant passages from your own content (website pages, FAQs, documentation) and then asks the model to answer using only that material. It is what allows a chatbot to answer accurately about a specific business.

Does RAG stop chatbots from hallucinating?+

It reduces hallucination substantially but does not eliminate it. Three failure modes remain: retrieval can return the wrong passage, your source content can itself be wrong or out of date, and a model can combine two accurate passages into a conclusion neither supports. A well-built system says "I don't know" when retrieval finds nothing relevant.

Is RAG better than fine-tuning for a customer service chatbot?+

For factual questions, generally yes. RAG updates when your website updates, can show which source produced an answer, and lets you fix a wrong answer by fixing the source. Fine-tuning requires retraining to update and cannot explain where an answer came from. Fine-tuning remains useful for tone and format rather than facts.

Do I still need to write question-and-answer pairs?+

No. That was the previous generation of chatbot tooling. A retrieval-based bot reads your existing website and drafts answers from it, so setup is closer to an afternoon than a project. Your work shifts to checking that your published content says what you want it to say.

How does the bot know which page to use?+

The customer's question is converted into a meaning-based search rather than a keyword match, so a question about "something for a wet room" can find a page that only says "bathroom". The closest-matching passages are retrieved and passed to the model with the question.

The summary

RAG means the bot looks things up before it speaks. That is worth understanding, because it tells you where to look when an answer goes wrong. A bad answer is usually a retrieval problem or a content problem rather than a model problem, and both of those you can actually fix.

Revdot is built on retrieval: enter your site address and Revdash reads your pages, builds the bot and emails you the link in 10 minutes. Nothing is installed on your site until you decide. The chat widget in the corner and the inline bot read the same knowledge base and share one quota and one dashboard, which also lists the questions the bot could not answer.

Try it on your site → See the chat widget

Prices are public: from 39 € a month for 300 conversations, and the inline bot is included in Pro, Business and Scale. Related reading: how to build a chatbot without writing code, or see what a subscription costs.

Try it on your site

Your own bot tells you more than any article.

Enter your site address, and Revdash builds a bot from your content and emails you the link in 10 minutes. Prices are public, from 39 € a month.

What is your main goal?

Revdash reads your site · knowledge base · bot in your inbox

The Revdot team answers in person:WhatsAppBook a call+358 45 231 1420eelis@revdot.ai