How to tell whether AI is actually improving outcomes in your contact center
By Chris Karp, Chief Growth Officer, Afiniti
I started my career as an agent. Funny enough, a lot of the metrics we used to run contact centers back then are the same ones we use today.
AHT. FCR. CSAT. NPS.
I don’t think they’re going away, and I don’t think they should. They all tell you something useful.
The problem starts when a metric stops being an indicator of success and becomes the goal itself.
Measurement drives behavior
If I tell an agent the most important thing is getting off the phone in six minutes, they’ll optimize for six minutes.
If I tell an AI system its job is to maximize containment, it’ll maximize containment.
Neither is doing anything wrong. They’re doing exactly what we asked. We just never connected those targets to resolution, retention, or revenue.
Now picture every part of the operation chasing something different:
- The AI agent is optimized for containment: no handoff to a person
- Routing is optimized for speed: the next available agent
- WFM is optimized for utilization: every agent kept busy
- The business wants retention: keep the customer from leaving
Every team can hit its target while the customer journey gets worse. And most organizations aren’t set up to see it. Today, only 22% of contact center leaders say their AI environment is fully integrated into an end-to-end strategy.
Contained isn’t resolved
Containment is a useful measure. It can lower operating costs and clear routine interactions quickly. The trouble starts when it becomes the only thing being measured, because the downstream effects go unseen.
Let’s say a customer is disputing a charge. They start in self-service. The AI can’t solve it, but it keeps trying because it was designed to maximize containment. Eventually, the customer gives up.
On the dashboard, that looks like a success. The conversation never reached an agent.
Then the customer calls back the next day with the exact same issue, and this time they want a person.
You haven’t eliminated the work. You’ve created another interaction, added frustration, and probably made the human conversation harder.
When I see that 49% of customers don’t see AI actively improving the customer experience, this is usually what I think leaders are missing.
My simplest advice is this: follow the customer, not the interaction.
- Did they come back an hour later?
- Did they call the next day?
- Did they switch channels and start over?
- Did they demand a human until they got one?
- Did they abandon without their issue resolved?
If they did, you didn’t remove the effort. You just moved it.
Customers already vote with their feet here. Only 17% would consider abandoning a phone call with a person, compared with 38% for a chatbot and 40% for a self-service portal or app.
And a handoff to a person isn’t a failure. If AI recognizes that a human is better placed to help and passes the context along, that can be a much better outcome than keeping someone contained for another five minutes.
“The goal shouldn’t be to keep the customer in AI. The goal should be to use AI to get the customer to the right outcome.”
Give every efficiency metric a partner
If I could make one practical change to most scorecards, it would be this: never look at an efficiency metric on its own.
| If you’re tracking… | Put this next to it | So you can see… |
| Containment | Resolution and same-intent repeat contact | Whether the issue was actually solved |
| AHT | FCR, CSAT, or retention | Whether speed came at the cost of the outcome |
| Cost per contact | Total contacts per issue | Whether savings in one place created work somewhere else |
Imagine a customer calls to cancel their service. The agent takes a few extra minutes to understand what’s really going on, fixes the underlying problem, and retains the customer.
Their handle time just got worse. The business outcome got significantly better. If AHT were the only measure, that call looks like a failure. Put FCR and retention next to it, and you can see what actually happened to the business.
Your scorecard needs to recognize that.
Three tests for whether AI is working
When someone tells me an AI investment is working, I want to see evidence in three areas:
- Customer outcomes. Are more issues getting resolved? Are repeat contacts coming down? Is effort dropping, in transfers, channel switches, and time to resolution?
- Operational impact. Did we reduce the total work needed to serve the customer, or just move it to another channel and add time to the customer’s journey?
- Business value. Did cost to serve go down? Did retention, conversion, or revenue go up?
I’d also be careful about calling success or failure too quickly. If a metric like containment swings up or down in a single week, that’s interesting. What I really want to know is what happens over the next several weeks. Are those customers coming back? Are they escalating somewhere else?
That kind of patience is hard right now. 91% of contact center leaders expect results from a technology investment within a year, and 13% expect them in under 90 days. I understand the pressure. But measure too early, and it’s easy to celebrate the wrong thing.
The cost of getting this wrong is real. 15% of leaders say they’ve abandoned at least one in four technology initiatives because they couldn’t prove value.
Where to start
You don’t have to transform the whole contact center at once. Please don’t try to boil the ocean.
- Pick one journey that really matters to the business.
- Define what a better outcome looks like for the customer and the company: resolve the issue, retain the customer, drive a sale, or reduce total cost to serve.
- Map everything that touches it: the AI, people, handoffs, data, rules, and metrics.
- Ask what each piece is optimizing for, and where the customer is repeating themselves.
- Agree on one shared outcome and set a baseline.
- Test changes, even manually at first. Prove you can move the outcome, then automate and scale.
That one journey will probably be a microcosm of what’s happening across the rest of your operation. It’s just much easier to solve one problem at a time.
Thoughtful beats fast
A lot of companies feel pressure to deploy more AI because the technology is moving quickly and nobody wants to be left behind.
But the companies that create the most value won’t be the ones that deploy the most AI. They’ll be the ones that deploy it most thoughtfully, around outcomes their customers and their business actually care about.
To me, that’s the real shift. It’s what we mean at Afiniti when we talk about outcome orchestration: getting the AI, people, data, and systems you already have to make better decisions together.
I explored these ideas with Audrey Steeves at CCW Digital’s Modernizing Service Experiences With AI & Digital event.
Want the research behind this? CCW Digital’s latest market study, produced in partnership with Afiniti, looks at what customers now expect from modern service, how leaders are preparing for a more AI-driven customer journey, and where AI and people each have a role.