How to Hire an Agentic AI Consultancy: Questions Every Contact Centre Leader Must Ask

Arkadas Kilic 9 min read · 2026-05-24

Rel8 CX is an AWS Advanced Partner that builds autonomous AI agents for regulated contact centres. We've seen what happens when contact centre leaders pick the wrong partner. This post exists so you don't make that mistake.

The market for agentic AI services has exploded. Every consultancy now has a slide deck about AI transformation. Most of them have never shipped a production agent. They've run pilots. They've built demos. They've presented findings. But when it comes to an autonomous AI agent handling live calls, making real decisions, and operating inside FCA or HIPAA guardrails at scale, the list of firms who've actually done it gets very short, very fast.

Here's what to ask before you sign anything.

The difference between a consultancy and a builder

Before we get to the questions, let's be clear about what you're actually buying.

A consultancy advises. They assess your current state, produce a roadmap, and hand you a document. Some of them are excellent. But if your goal is a production AI agent answering calls, processing claims, or handling debt repayment conversations by a specific date, you don't need advice. You need someone who builds.

The distinction matters because the evaluation criteria are completely different. You wouldn't hire an architect to lay bricks. Don't hire a strategy firm to deploy infrastructure.

Ask every firm you speak to: "What did you ship last quarter, and can I speak to that client?"

The answer tells you everything.

Question 1: What does your production track record look like?

Not pilots. Not proofs of concept. Not "we worked with a major bank on an AI initiative."

Production means: live, handling real customer interactions, in a regulated environment, without a human reviewing every response before it goes out.

Ask for:

• The number of production deployments completed in the last 12 months
• Average time from contract to go-live
• A reference call with a client in a similar industry

At Rel8, we deploy production agents in 4 to 6 weeks. That's not a marketing number. It's the actual timeline from our last several deployments. If a vendor can't give you a specific number, that's your answer.

Question 2: How do you handle compliance in a regulated contact centre?

This is where most vendors fall apart.

In financial services, healthcare, utilities, and collections, compliance isn't a feature you add at the end. It's architecture. It has to be baked into every layer: how the agent decides what to say, what it refuses to say, how interactions are logged, how audit trails are structured, and how you demonstrate compliance to a regulator if challenged.

Ask specifically:

• How does your architecture enforce FCA Consumer Duty (or HIPAA, or your relevant framework)?
• Where are interaction logs stored and for how long?
• Can you show me the guardrail layer in your agent architecture?
• Have you been through a compliance review with a regulator or internal audit team? What was the outcome?

Vague answers about "built-in compliance features" are a red flag. You want to hear about specific controls: prompt-level guardrails, PII redaction pipelines, immutable audit logs in S3, role-based access to recordings, and a clear data residency position.

We build on AWS natively. Every deployment uses AWS-native services for logging, encryption, and access control. That's not a preference. It's a compliance requirement for most of our clients.

Question 3: Are you AWS native, or are you bolting a third-party platform onto AWS?

This matters more than most buyers realise.

AWS native means your agent runs on services like Amazon Connect, Amazon Bedrock, AWS Lambda, and Amazon DynamoDB. Your data doesn't leave the AWS boundary. Your security posture is consistent. Your cost model is predictable. Your team can manage it without learning a proprietary platform.

Third-party platforms bolted onto AWS introduce a layer of abstraction that creates vendor lock-in, adds latency, complicates compliance, and makes your architecture harder to audit.

Ask:

• Which specific AWS services does your architecture use?
• Is there any third-party middleware sitting between our contact centre and the AI layer?
• If we wanted to move away from your firm in 18 months, what would that look like?

The last question is particularly revealing. A builder who's confident in their work will give you a straight answer. A vendor protecting a proprietary platform will get uncomfortable.

Question 4: What does your team actually look like?

Not the org chart. The people who will be on your account.

Ask for CVs or LinkedIn profiles of the engineers who will build your agent. Look for:

• AWS certifications (Solutions Architect, Machine Learning Specialty, Developer)
• Evidence of previous contact centre deployments
• Hands-on coding experience, not just architecture diagrams

A common pattern in this market: a senior practitioner sells the engagement, then a team of junior consultants delivers it. The person who understood your problem is now selling to someone else.

At Rel8, the engineers who scope your deployment are the engineers who build it. Our team is small by design. Every person on a project has shipped production agents before.

Question 5: How do you define and measure success?

This is a commercial question, not a technical one. But it tells you a lot about how a vendor thinks.

Bad answer: "We'll track agent utilisation and containment rate."

Better answer: "We'll define a baseline for your current cost-per-interaction, set a target for automated containment, and measure against it weekly from week two onwards."

The best answer includes specific numbers from comparable deployments. We've seen 47% containment in the first week of a live debt collections deployment. We've seen average handle time drop from 6 minutes 40 seconds to under 2 minutes for routine balance enquiries. These aren't projections. They're numbers from actual builds.

Ask:

• What metrics did you hit on your last three deployments?
• How do you handle it if we miss the target?
• What's your escalation path when the agent fails?

Question 6: What's your escalation and fallback architecture?

Every production AI agent will encounter situations it can't handle. The question isn't whether that happens. It's what happens next.

A well-designed agent has:

• A defined confidence threshold below which it escalates to a human
• A graceful handoff that passes full context to the agent, not just a warm transfer
• A fallback flow that doesn't leave the customer stranded
• Monitoring that flags unusual escalation rates so you can tune the model

Ask to see the escalation architecture. Ask what the average escalation rate was on their last deployment. Ask how long it took to tune the agent to reduce unnecessary escalations.

If they don't have a specific number for escalation rate, they haven't built a production agent.

Question 7: What does the 4 to 6 week timeline actually include?

Timeline claims are everywhere. What matters is what's in scope.

Ask for a breakdown:

• Week 1: Discovery, integration scoping, environment setup
• Week 2 to 3: Agent build, prompt engineering, integration with your telephony and CRM
• Week 4: Internal UAT, compliance review, load testing
• Week 5 to 6: Soft launch, monitoring, tuning

Then ask what's NOT included. Common exclusions:

• CRM integration (often scoped separately)
• Custom reporting dashboards
• Multi-language support
• Post-launch optimisation beyond 30 days

A vendor who gives you a 4-week promise without a clear scope is setting you up for a change-order conversation at week three.

Question 8: How do you handle model updates and drift?

AI models change. The underlying foundation models get updated. Your contact centre processes change. Regulatory requirements shift. A production agent that worked well in January can start producing poor outputs by April if nobody's watching.

Ask:

• How do you monitor for output drift?
• What's your process when a model update affects agent behaviour?
• Is post-launch monitoring included in the engagement, or is it a separate retainer?

This is an area where the difference between a builder and a consultant is stark. A builder has a monitoring stack. A consultant writes a recommendation.

Question 9: Can you show me the actual architecture diagram?

Not a marketing diagram with rounded boxes and arrows. The real one. Services, data flows, integration points, security boundaries.

A firm that has built production agents has this document. It's part of their delivery. They can share a sanitised version without revealing client data.

If they can't produce an architecture diagram in the sales conversation, they haven't built what they're claiming to have built.

Question 10: What's your position on agentic AI versus automation?

This is a litmus test for how the vendor thinks.

Automation follows rules. If the customer says X, do Y. It's deterministic. It's been around for 20 years. It's fine for simple, predictable flows.

Agentic AI reasons. It interprets intent, retrieves information from multiple sources, makes decisions based on context, and takes actions. It can handle the 40% of interactions that don't fit a neat script.

A vendor who conflates the two is either confused or selling you something older than they're admitting.

Ask: "Where does your architecture stop being automation and start being genuinely agentic? Can you show me an example?"

The answer should include a specific example of an agent making a non-deterministic decision: retrieving a customer's account status, cross-referencing it against a policy, and offering a resolution that wasn't pre-scripted.

Question 11: What does your pricing model look like, and what are the total costs?

Pricing in this space is opaque by design. Here's what to ask for:

A transparent vendor will give you a total cost of ownership estimate for year one, including AWS infrastructure. A vendor who only quotes the build fee is hiding the real number.

For context: a production Amazon Connect AI voice agent handling 10,000 interactions per month typically runs between $3,000 and $8,000 per month in AWS infrastructure costs, depending on call volume, agent complexity, and the number of tool calls per interaction. That number should be in your business case from day one.

Question 12: Why should I trust you with a regulated, customer-facing deployment?

Ask this directly. The discomfort it creates is informative.

A builder who has done this before will answer with specifics: the industries they've worked in, the compliance frameworks they've navigated, the incidents they've handled and how, and the clients who will vouch for them.

A consultancy who hasn't will pivot to credentials, case studies from adjacent industries, and assurances about their process.

There's nothing wrong with a firm that hasn't done your exact use case before. But they should be honest about it. And you should price that risk accordingly.

A note on red flags

After running these evaluations with clients across financial services, utilities, and healthcare, here are the patterns that consistently precede a bad outcome:

What good looks like

A firm worth hiring for a production agentic AI deployment in a regulated contact centre will:

• Show you a live agent call from a previous deployment (with client permission)
• Give you a reference in a comparable industry
• Produce an architecture diagram on request
• Quote a 4 to 6 week timeline with a clear scope
• Name every AWS service in their stack
• Have a specific answer to every compliance question
• Tell you what the agent can't do, not just what it can

That's a high bar. It should be. You're putting this agent in front of your customers.

Who is Rel8 CX?

Rel8 CX is an AWS Advanced Partner. We build autonomous AI agents for regulated contact centres in financial services, utilities, healthcare, and collections. We deploy to production in 4 to 6 weeks. Our team builds. We don't advise.

If you're evaluating partners for an agentic AI deployment, we're happy to answer every question on this list.