Chatbot Analytics and the Metrics Worth Tracking Each Week
Your website chatbot has been live for a month, the dashboard says it handled 1,400 conversations, and you still cannot tell whether it helped anyone.

Your website chatbot has been live for a month, the dashboard says it handled 1,400 conversations, and you still cannot tell whether it helped anyone. Most chatbot analytics screens are built to make the bot look busy, not to show you where it fails.
This guide covers the handful of numbers that actually tell you if a bot is working, how to calculate them from the conversation logs most platforms export, and a 30 minute weekly review that turns them into fixes. It is written for small teams who own the bot themselves, not for a data department.
Why most chatbot analytics dashboards mislead you
Platforms lead with volume: total conversations, messages sent, unique visitors. Those numbers go up when the bot is helpful, and they also go up when the bot is confusing and people have to rephrase three times. A conversation where a customer types "agent agent AGENT" counts the same as one where the bot answered a delivery question in two messages.
The second problem is how "resolved" is defined. On many tools a conversation counts as resolved if the visitor simply stopped typing. People stop typing when they get an answer, but also when they give up and pick up the phone. Without separating those two, your resolution rate is close to meaningless.
The six metrics worth tracking
You do not need twenty charts. These six cover almost every useful question about a small business bot.
| Metric | How to calculate it | Healthy range to aim for |
|---|---|---|
| Fallback rate | Conversations with at least one "I didn't understand" reply, divided by all conversations | Under 15 percent |
| Handoff rate | Conversations passed to a person, divided by all conversations | 10 to 30 percent, depending on your business |
| Confirmed resolution | Conversations ending with a thumbs up or "yes, that helped" | Rising month on month |
| Repeat contact | Visitors who contact you again by any channel within 7 days about the same issue | Under 10 percent |
| Drop off point | The bot message after which most abandoned conversations end | No single message above 20 percent of drop offs |
| Top unanswered questions | The 10 most common messages that triggered a fallback | A list that changes every few weeks |
Fallback rate
This is the easiest signal to get and the most honest. Every time the bot says it did not understand, a visitor hit a gap. If your rate is above 25 percent, the bot is guessing on a quarter of its traffic. Improving the reply itself helps a little, and our chatbot fallback message examples show how, but the real fix is closing the gaps those fallbacks reveal.
Handoff rate
A low handoff rate is not automatically good. If your bot never passes anyone to a person, it may be trapping people who need one. For a shop answering order status questions, 10 percent may be right. For a clinic or a law office, 30 percent can be healthy. What matters is whether handoffs happen at the right moments, which we cover in when a chatbot should hand off to a human.
Confirmed resolution and repeat contact
Add a one-tap question at the end of each answer: "Did this solve it?" with yes and no buttons. Only around 15 to 25 percent of visitors will tap anything, but that sample is far more honest than silence. Pair it with repeat contact. If someone used the bot on Tuesday and emailed support on Wednesday about the same order, that conversation was not resolved, whatever the dashboard says. Matching them needs an email or order number, so this works best for logged-in or post-purchase chats.
Containment rate is the wrong goal
Many vendors and consultants push "containment" or "deflection" as the main chatbot number: the share of conversations that never reached a human. It sounds like saved staff time, so it ends up on slides and in targets.
I think it is the most harmful metric you can optimise. The easiest way to raise containment is to make the human option harder to find. Hide the "talk to someone" button behind three menus and containment jumps from 70 to 85 percent in a week. Meanwhile the customers you blocked call your phone line angrier than before, or leave for a competitor, and none of that appears on the bot dashboard.
Track containment if your platform shows it, but never set it as a target on its own. Read it only next to repeat contact and confirmed resolution. If containment rises and repeat contact rises with it, the bot got worse, not better.
A 30 minute weekly chatbot review
Numbers only help if someone looks at them on a schedule. Here is the routine I use with small clients, usually on Monday morning.
- Five minutes: check the four rates. Write fallback, handoff, confirmed resolution and repeat contact into a simple spreadsheet, one row per week. You are looking for change, not perfection.
- Ten minutes: read 10 failed conversations. Filter for fallbacks or thumbs down and read them in full. Do not skim. You will usually spot the same missing answer three or four times.
- Five minutes: check the drop off point. If many people abandon after the same bot message, that message is too long, asks for too much, or offers the wrong buttons.
- Ten minutes: fix one thing. Add one missing answer, rewrite one confusing message, or add one handoff trigger. One fix a week adds up to 50 a year.
The fix almost always lands in the knowledge base. If you built your bot from real customer questions, as described in chatbot knowledge base built from questions customers ask, each new unanswered question fits straight into that list.
Getting the data out of your platform
Most chatbot tools for small businesses let you export conversation logs as a CSV file, even on cheaper plans. Look for columns such as conversation ID, timestamp, sender, message text, and any intent or "matched answer" field. With those you can calculate every metric above in a spreadsheet using basic filters and COUNTIF.
A few practical tips. Remove your own test conversations first, because a week of testing can be 10 percent of early traffic. Look at weekday and weekend separately, since weekend visitors often have different questions. And keep raw exports for at least three months so you can compare before and after a change.
Frequently asked questions
What is a good resolution rate for a chatbot?
It depends on what counts as resolved. Using confirmed resolution, where the visitor says the answer helped, small business bots often sit between 40 and 70 percent. Any figure above 90 percent usually means the platform is counting silence as success.
How often should I review chatbot analytics?
Weekly for the first three months after launch, when gaps are easy to find and fix. After that, every two weeks is usually enough, with a deeper monthly look at trends. Reviewing daily tends to make people react to noise.
What does fallback rate mean in a chatbot?
Fallback rate is the share of conversations where the bot could not match a message to any answer and replied with a default "I didn't understand" message. It is the clearest sign of gaps in what the bot knows. A rising fallback rate often means customers have started asking about something new.
More in Chatbots
Chatbots
Chatbot Privacy and What Your Bot Should Not Store
I read fifty transcripts from a small physiotherapy clinic's website bot last spring, as a favour.
Chatbots
Chatbot Buttons vs Free Text and When to Use Each
A friend who runs a two person accountancy practice replaced her old button menu with a shiny new text box last spring, because...
Order Tracking Chatbot for a Small Online Shop
If you run a small online shop, a large share of your support inbox is probably one question asked fifty ways: where is my...