AI-powered chatbots were supposed to be the panacea for customer service: available 24/7, efficient, and responsive. However, the reality is sometimes quite different. Between unexpected answers, dubious advice, and unintentional prompts to switch to competitors, these virtual agents can become a real headache for brands.
How can you maintain control while still leveraging what they can offer? This question deserves attention, especially since behind the difficulties might lie in an opportunity you hadn’t considered.
Everything was going well. Until that answer.
AIs don’t always react as expected. It’s a problem we’re familiar with, but whose consequences we sometimes underestimate.
We adapt to it when using an assistant ourselves. It’s more problematic when a chatbot installed on your site becomes counterproductive, explains how to circumvent some of your rules, even worse, sends your customers directly to your competitors…
Unfortunately, anticipating all requests and adapting to them is almost impossible.
This topic is discussed by Yajush Gupta in his article on the risks of AI agents for businesses, published on Industry Contents. He describes chatbots that are sometimes difficult to control, even in large companies.
He cites the particular case of a banking chatbot that explained to customers how to avoid certain fees, or that of a car dealership whose chatbot recommended a vehicle from a competing brand.
This was probably not in the brief.
The detail that was missing from the promise
Companies have long been encouraged to offer chatbots to their customers: to relieve sales teams, assist with purchases, or answer after-sales service questions.
But are they all aware of the risks? And do the service providers who sell them these services explain them sufficiently?
An erroneous answer can have very real consequences: disappointing a customer, creating a misunderstanding about an offer, or involving the company in a dispute.
This is precisely what happened in the Air Canada chatbot case. A traveler had received an incorrect information about the possibility of requesting a bereavement discount after their trip. The company then refused this refund. In February 2024, the tribunal held Air Canada responsible for the information provided by its chatbot and awarded compensation to the customer.
Therefore, becoming aware of the problem is necessary. But that’s only the first step.
What happens when you’re not looking?
A first approach is to analyse the questions addressed to the chatbot and the answers it provides.
The idea: use an AI tool to examine these exchanges and flag unexpected or potentially problematic cases for the company.
This analysis will not necessarily correct the conversation in real-time. However, each anomaly identified can help refine the chatbot’s instructions, complete its information, or uncover a situation that no one had anticipated.
Sometimes, the answer will not only be technical. A wrongly handled client request can also reveal an ambiguous rule, a missing information or a company policy that needs to evolve.
The chatbot then becomes a revealer of an already existing problem, without being as visible.
Your clients are always one-step ahead
At human scale, it is impossible to simulate every possible situation or conversation your clients will have with your chatbot.
However, AI can help you expand the range of test scenarios.
Unusual requests, ambiguous demands, objections, misuse attempts: it can offer a new vision on cases your team might not have thought about.
The goal isn’t to claim you can imagine every possibility. It is to prepare the chatbot to wider range of interactions and recognise its difficulties before it is released.
Even then, the AI is an adviser. It enriches your thinking, without replacing the knowledge of clients and the job your teams have.
At what point can we let it improvise ?
These two approaches lead to an essential project: building a diverse and reliable database to feed your chatbot.
This database has to be tested and updated by humans: specialists, business and marketing teams and customer services.
Which claims can the chatbot make ? On what information can it rely on ? In which cases does it have to realise it can not do the job et let a human do it ?
Such a database cannot ensure there won’t be any abuses on its own. But, with the right instructions and constant follow up, it can prevent improvisation.
And, this structurational work may have another use.
What you’re looking for is already in your hands
I feel like such a project possesses an additional merit widely unexploited?
As I have said before on OlivierSauvage.com, companies have gold in their hands: the content they are making, as well as daily exchanges with clients and prospects.
A precise question. A recurring objection. An answer that solves a problem. A use case no one has ever thought about.
This exchanges have something your opponents do not necessarily have yet.
The only criteria is to respect confidentiality and personal data, you can make articles , illustrations or videos that answer some of your clients concerns.
Rarity is not the only valuable thing you can get out of those. What is interesting are practical experiences, a precise answer, knowledge that is hard to find.
That, is what can make the data useful for your readers and what might be cited by search engines and AI assistants.
Key idea: exchanges with client are a base material to create specific content, useful and rooted in your experience. Which means you need to take time to spot and exploit this material.