AI chatbot for customer service: when to do it, when not to

80% of the questions to an average customer service desk are repetition: opening hours, delivery status, how does this work, where do I find that. Those are exactly the questions an AI chatbot can catch, provided it is built well. But there is an important condition, and it is often skipped: the bot has to draw answers from your own information, not from what a language model happens to "think" it knows.

The difference between an ordinary chatbot and RAG

A language model like GPT or Claude knows a lot, but nothing specific about your company. Ask a bare chatbot about your returns policy and it invents a plausible-sounding answer that is wrong. That is called hallucinating, and it is deadly for customer service.

The solution is RAG, Retrieval-Augmented Generation. In plain terms: before the AI answers, the system first searches your own documents, manuals and FAQs for the relevant passages. It passes those to the language model with the instruction: answer the question only based on this information. That way the answer comes from your source and not from the model's imagination.

For Lexi AI we built exactly this for collective labor agreement text: the assistant searches hundreds of pages of agreements and gives a substantiated answer with a reference to the right passage. That is RAG in action, and it is the difference between a usable and a dangerous chatbot.

When an AI chatbot does pay off

A chatbot is a good investment when:

• You get many recurring questions whose answers are fixed
• You have a decent knowledge base or documentation, or are willing to create one
• Customers want answers outside office hours
• Your customer service team is drowning in simple questions and has no time for the hard ones

In those cases the bot catches the bulk of the volume and your people have time left for the conversations that truly matter.

When you shouldn't do it

Be honest: a chatbot is no miracle cure. Don't do it if:

• Your questions are almost always unique and complex
• You have no usable knowledge base and don't want to build one
• Your customers mainly discuss emotional or sensitive matters, think complaints or care
• You expect a bot to replace your entire customer service, because it won't

A badly deployed chatbot frustrates customers more than it helps. The notorious "I didn't understand that, could you phrase it differently" is exactly what you want to avoid.

What does an AI chatbot cost?

There are two kinds of cost: build and usage.

• Build costs: between €8,000 and €35,000, depending on complexity, integrations and how many knowledge sources have to be ingested
• Usage costs: per conversation you pay for the tokens the language model processes. With good architecture and caching we keep this low, often a few cents per conversation

The biggest hidden cost is getting your knowledge base in order. Good answers require good sources. In practice a lot of time goes into that, and you have to plan for it in advance.

The biggest pitfalls

Our projects keep surfacing the same lessons:

• No escape to a human: always build in a clear route where the customer reaches a person when the bot can't work it out
• Allowing hallucinations: without tight RAG instructions the model invents things. Test this thoroughly
• An outdated knowledge base: a bot is only as good as its sources. Keep your documentation up to date, otherwise it gives old answers
• Wrong expectations: tell customers they are talking to an AI and what it can and can't do

Our approach

We build on Claude, GPT or Mistral, depending on what best fits your requirements for quality, cost and privacy. We build the RAG layer so the bot draws only from your sources and says a clean "I don't know, I'll put you through" when the answer is missing. Better an honest "I don't know" than a confident lie.

Which language model do you choose?

There is no single best model, it depends on your requirements. Claude, GPT and Mistral each have their strengths in quality, cost and privacy. For a customer service desk that works with personal data, processing within Europe weighs heavier. For a simple FAQ bot you mainly watch the cost per conversation. We pick the model based on your situation, and build it so we can switch later if another model turns out to be better or cheaper. So you are not tied to a single vendor.

Start small and measure

The best approach is not to automate your entire customer service straight away, but to start with the twenty most-asked questions. You cover those, you put the bot live, and you measure what happens:

• What percentage of questions does the bot handle on its own?
• Which questions does it refer to a human, and is that the right call?
• Where do customers get frustrated or drop off?

Based on that data you extend. That way you avoid building for months on answers to questions that are barely asked, and you adjust to what customers actually ask. A chatbot, like all good software, is something you grow based on real usage instead of assumptions.

Considering an AI chatbot? Take a look at our AI service. We first check whether it pays off in your case at all, before we build anything.