What actually breaks in an AI phone agent
Twenty-four defects, every one found on a real call rather than in testing. This is the record — what broke, what it cost, and the guard that now catches it.
Most AI receptionists are demonstrated, not tested. They handle the call the salesperson scripts and fall over on the one nobody rehearsed — the caller who mishears a question, the one who won’t give a number, the one asking about something the business may not even do.
We run a voice agent on a live line in Toronto and keep a written record of every failure. Four of them are below, in full. None are hypothetical, and none were caught by a demo.
It booked a two-day job into a two-hour slot
A caller asked about a ceramic coating — multi-day work that a shop schedules itself around other jobs. The agent booked it as a standard two-hour appointment.
A car arrives for work nobody has time to do, and a customer expects it back that afternoon.
The instructions already said not to do this. The agent had been told, in plain language, that detailing must never be booked. It booked it anyway, because the route to the booking step still existed and the model took it.
We deleted the route. Quote-only work now flows to a step with no connection to booking at all. It isn’t that the agent is told not to book a ceramic coating — it’s that there is nowhere for the booking to go.
Guarded by scenario S3, which fails if the path is ever reconnected.
It read back a phone number from its own instructions
The transcript at the top of this page. The confirmation step carried a real phone number as an example of how to read one aloud. The agent recited the example instead of the caller’s number, the caller corrected it, and the booking then failed against the wrong number.
The customer had to argue with a machine about their own phone number, then lost the appointment anyway.
Earlier sweeps missed it because they searched for the digits. The number was written out in words.
No literal example may appear anywhere a model could say it out loud. Examples now read <name> and <their number>, which teach the same shape and cannot be lifted. A quoted string in an instruction is indistinguishable from a line to deliver.
Guarded by scenario S16, which fails on any part of the old number.
It turned away a customer for work the business does
Asked whether the shop replaced windshields, the agent answered “we don’t do windshield replacement.” Six times out of six.
Nobody had ever told it that. There was no list of declined services anywhere — only a price list. The agent treated an absence from a price list as proof of scope and stated the inference with complete confidence.
A paying customer sent elsewhere, and something untrue said about a business we don’t own.
An absence from a price list is a pricing gap, never a no. The agent now says plainly that it isn’t sure and won’t guess, then offers a person — a transfer if the business is open, a callback if it isn’t, decided by comparing the clock against opening hours rather than by judgement.
Fabrication measured 6/6 before the fix and 0/6 after.
It said goodbye three times and never hung up
The closing step could only end the call once a goodbye had been spoken. The agent kept re-emitting the wrap-up instead, so the exit was never satisfied and the loop had no floor.
A booking that went perfectly, undone in the last ten seconds.
The close now speaks exactly once, and that single message contains the goodbye. The trailing question became a statement — “anything else, just give us a ring” — so there is no second turn to loop on. A first attempt moved the goodbye into the end step instead; that step cannot speak, and calls ended in silence. Both are in the record.
Guarded by an assertion that fails any run where an identical utterance appears twice.
Two patterns behind most of them
1. Instructions lose to structure
A step says do X, the route out of it permits leaving without X, and the route wins. Four separate defects took this shape. Telling a model not to do something is the least reliable instrument available — and when the prohibition was written more forcefully, it over-applied and the agent stopped asking for names at all.
2. Examples get said out loud
Three defects, including the phone number above. A model cannot reliably tell an illustration from a line of dialogue. If an example must never be spoken word for word, it cannot be written as a quote.
Neither is specific to any trade. We found them running an auto shop line because that is the line we run; the same failures appeared independently in a home-services agent built six months earlier, before any of this was known. What generalises is the behaviour underneath — how contact details are collected, how a call ends, what happens when the agent doesn’t know something. The vocabulary changes by trade. The failures don’t.
What this testing still cannot see
Sixteen scenarios run against the agent before any change goes live. They are not sufficient, and pretending otherwise is how the next defect reaches a customer.
The first thing the agent says is not faithfully testable.
One guardrail measured a clean six out of six in simulation on the exact version that was skipping it on one live call in three. Anything on the opening turn gets checked by dialling the number.
Calendar tools are simulated.
The real booking boundary is never exercised by the suite, so a booking is made by hand, through to confirmation, after any change that could touch it.
Two behaviours are probabilistic and recorded as such.
Measured rates, published in the record, rather than described as fixed.
A green test suite is evidence, not proof. The record exists so that the difference stays visible.
Questions we get asked
Do AI receptionists actually work?
The technology works. Most deployments don't, because they are configured once and never tested against the calls that go sideways. The difference between an agent that helps and one that costs you customers is entirely in the failures somebody bothered to find.
What goes wrong with AI phone agents?
In our record: booking work that can't be booked, repeating a question until the caller hangs up, inventing prices or availability, claiming a business doesn't offer something nobody said it didn't, promising callbacks that were never agreed, and reciting fragments of its own instructions as though they were the caller's details.
How do you know it won't book the wrong thing?
Because on the paths where booking would be wrong, there is no booking step connected. That is a structural guarantee rather than an instruction the model can override, and it is the pattern we reach for whenever a mistake would be expensive.
Will it say something untrue about my business?
It is restricted to facts you supply — your hours, your address, your prices. Anything outside that, it says it isn't sure and offers a person. The hardest version is the question that sounds answerable, like whether you handle a service that isn't on your price list. Case 03 below is exactly that failure and how it was closed.
Can it put someone through to a real person?
Yes, and the choice is made by the clock rather than by judgement: a transfer while you're open, a callback when you're closed, because a transfer that rings out in front of a caller is worse than not offering one. Transfers are verified end to end on a real handset before a line goes live.
Is this only relevant to auto shops?
No. The record came from an auto shop line because that is the line we run, but nothing in it is about cars. The same behaviours — how a call ends, how contact details are collected, what the agent does when it doesn't know — carry across bin rental, heating and cooling, electrical and any other trade where the phone is the front door. Each vertical adds its own rules on top; the underlying failures are shared.
How long does it take to get one running?
Building the agent is the fast part. The work is deciding which call path is actually costing you business, configuring around how you already operate, and testing the edges before it answers a customer. That is what the Digital Front Door Review establishes first.
Every figure on this page comes from our own QA record, maintained since the first live call. No third-party research is cited, because none of it is about this agent.
Hear it for yourself
The demo line runs the agent this record describes. Ask it something awkward — a price it doesn’t have, a service it’s never been told about, or just refuse to give your number and see how it ends the call.
Ninety seconds. No form, nothing to sign up for.
NorthTen · Toronto, Ontario
Digital front door systems for phone-driven local service businesses across the GTA and North America.
QA record maintained continuously since the first live call.