Red Flags in AI Voice Agent Vendor Demos: What Contact Centre Leaders Should Challenge Before Signing

Arkadas Kilic 7 min read · 2026-08-01

Vendor demos are theatre. The lighting is perfect, the script is rehearsed, and the AI never mishears a word. Then you go live, and reality arrives fast.

After building production AI voice agents for contact centres across regulated industries, I have sat through enough vendor pitches to know exactly where the gaps hide. This post gives contact centre leaders a direct, practical framework for separating production-ready systems from polished proof-of-concepts dressed up as enterprise software.

Challenge every one of these points before you sign anything.

1. The Demo Environment Is Not Your Environment

The single most common sleight of hand in AI voice agent demos is the controlled environment. Vendors demo over a clean VoIP connection, with a native English speaker, reading from a predictable script, with no background noise.

Your contact centre handles accented speech, poor mobile connections, customers talking over the agent, and calls that go nowhere near the expected flow.

• Ask to run a live demo against your actual call recordings, not their sample audio.
• Request the Word Error Rate (WER) figures from a real customer deployment, not a benchmark dataset. Industry-grade ASR should achieve WER below 10% on clean audio. Ask what it looks like on noisy or accented calls.
• Ask specifically: what happens when the caller says something completely outside the expected intent set? Watch how gracefully it fails.

If a vendor cannot demo against real-world audio, that tells you everything about where their system actually performs.

2. "Fully Autonomous" Usually Means "Autonomous in the Demo"

Vendors love the word autonomous. What it frequently means is: autonomous when the caller follows the happy path.

True autonomy in a contact centre context means the agent handles interruptions, topic switches, corrections, clarifications, and escalations without falling over. It means the agent can complete a transaction end-to-end, including backend system writes, without a human in the loop.

• Ask for the containment rate from a live deployment, not a pilot. A credible production system should achieve 60% to 80% containment on targeted intents. If a vendor quotes above 90% without qualification, push hard on how they define containment.
• Ask: does containment include calls where the customer hung up in frustration? Some vendors exclude abandoned calls from their containment calculations.
• Ask to see a call where the agent failed and how it handed off. A clean, low-friction escalation is a feature. Watch how they handle it.

3. Compliance Is an Afterthought, Not Architecture

For any contact centre operating in financial services, healthcare, insurance, or utilities, compliance is not a checkbox. It is architecture. If a vendor cannot explain exactly how their system handles data residency, call recording consent, PCI scope, and audit logging, you are looking at a system that was not built for your industry.

• Ask where call audio and transcripts are stored, and in which AWS region or cloud environment. If the answer is vague, that is a red flag.
• Ask whether their system is PCI DSS compliant for DTMF capture of card data, and whether voice capture is paused during sensitive input. This is non-negotiable for payments.
• Ask for a data flow diagram that shows every point where customer data touches their infrastructure.
• Ask whether their system supports call recording consent prompts that are dynamically controlled, not hardcoded.

Compliance built as an afterthought means your legal and security teams will spend months retrofitting controls. We build compliance into the architecture from day one, because going back is always more expensive.

4. The Integration Story Is Suspiciously Simple

In demos, the CRM updates instantly, the knowledge base returns perfect answers, and the backend system responds in milliseconds. In production, APIs time out, authentication tokens expire, and legacy systems behave in ways nobody documented.

• Ask how the agent handles a backend API timeout mid-call. Does it fail gracefully? Does it tell the customer something sensible? Does it log the failure for retry?
• Ask what their average integration timeline looks like for a contact centre with three or more backend systems. If the answer is under two weeks with no caveats, be sceptical.
• Ask whether they have direct experience integrating with your specific CRM or core system. Salesforce, Genesys, ServiceNow, and legacy telephony stacks all have integration patterns that require real production experience to navigate.

A vendor who has never integrated with your stack in a regulated environment is going to learn on your contract.

5. Latency Numbers Are Measured in Ideal Conditions

End-to-end latency is the metric that makes or breaks voice AI. Customers tolerate roughly 1.5 seconds of response latency before the interaction starts to feel broken. Most vendors quote latency figures measured from a server in the same data centre as their demo environment.

• Ask for P95 latency from a live deployment, not average latency. Average numbers hide the tail. P95 latency tells you what 1 in 20 callers actually experiences.
• Ask how latency is affected when the agent needs to make an outbound API call during the conversation. A real-world response that requires a CRM lookup adds 300ms to 800ms depending on the system. That needs to be accounted for.
• Ask whether their system uses streaming synthesis or batch synthesis. Streaming text-to-speech can reduce perceived latency by 400ms to 600ms. If they are using batch, that is a design choice with a real user experience cost.

6. The Pricing Model Obscures the Real Cost

Per-minute pricing sounds simple until you do the maths at scale. A contact centre handling 500,000 minutes per year at $0.08 per minute is paying $40,000 annually just in consumption fees, before platform licences, professional services, or support.

• Ask for a total cost of ownership model at your actual call volume, not a starter tier. Ask what happens to pricing at 2x your current volume.
• Ask whether there are fees for failed calls, calls that escalate to a human agent, or calls under a minimum duration threshold.
• Ask what is included in the base platform fee versus what is billed as professional services. Ongoing tuning, model retraining, and prompt updates are often excluded from headline pricing.
• Ask about contract exit terms. If a vendor is confident in their product, exit clauses should be reasonable. Aggressive lock-in terms are a signal.

7. The Go-Live Timeline Is a Guess

Every vendor promises a fast deployment. The realistic question is not how fast they can demo, but how fast they can deliver a production system that passes your UAT, satisfies your security team, and handles your real call volume.

• Ask for a reference customer in your industry who went from contract signature to production go-live in the timeline they are quoting you.
• Ask what the dependencies are on your side. Vendors who have done this before will have a clear list of what they need from you and when. Vendors who have not will give you a vague project plan.
• Ask what the most common reason is that their deployments run late. The answer is revealing. If they say "customer delays," ask how they manage that risk. If they say "integration complexity," ask how they handle it.

We build production AI voice agents in 4 to 6 weeks. That timeline is based on doing this repeatedly in regulated environments, not on optimistic project planning.

8. There Is No Post-Go-Live Story

The demo ends at go-live. But production AI voice agents require ongoing monitoring, intent drift correction, failure analysis, and continuous improvement. Ask what happens after launch.

• Ask how they monitor for intent drift, where caller language evolves away from the training data over time. This is a real production problem that typically surfaces within 60 to 90 days of launch.
• Ask what their SLA is for a production incident where the agent is mishandling calls at scale. Hours matter when thousands of callers are affected.
• Ask who owns the ongoing prompt engineering and model tuning. If the answer is "you do," make sure you have the internal capability to execute that.
• Ask for a sample of the monitoring dashboards and alerting they provide in production. If they cannot show you real operational telemetry, they are not thinking about production.

A Framework for Your Next Vendor Conversation

Before your next demo, bring this list. A vendor who is genuinely production-ready will welcome these questions. A vendor who deflects, qualifies excessively, or cannot provide live deployment references for any of these points is showing you exactly what your post-go-live experience will look like.

The questions that matter most:

1. Show me P95 latency from a live deployment in my industry.

2. What is your containment rate definition, and does it include abandoned calls?

3. Walk me through your data flow and where customer data is stored.

4. Give me a reference customer who went live in the timeline you are quoting.

5. What does your post-go-live monitoring and support model look like?

These are not trick questions. They are the baseline for enterprise-grade production systems.

What Production-Ready Actually Looks Like

At Rel8 CX, we are practitioners who build, not consultants who advise. Every system we deliver is AWS native, built with compliance requirements as architecture constraints from day one, and designed to go into production in 4 to 6 weeks. We work in regulated industries because that is where the standards are highest and the cost of getting it wrong is real.

If you are evaluating AI voice agent vendors and want a straight conversation about what production readiness actually requires, we are ready to have it.