Amazon Connect now provides eight new metrics to measure and improve AI agent performance

Amazon Connect AI Agent Performance Metrics

Amazon Connect now offers eight new metrics to evaluate and enhance AI agent performance, including goal success rate, faithfulness score, and tool selection accuracy. These metrics provide insights into the quality of AI-driven customer interactions, allowing for the measurement and continuous improvement of AI agent outcomes.

With this update, you can monitor if AI agents successfully resolved customer requests, assess faithfulness and detect contextual hallucinations, evaluate tool selection and utilization accuracy, and capture customer feedback through thumbs up/down ratings when enabled.

You can access these new metrics via the Amazon Connect AI Agent Performance dashboard, the GetMetricDataV2 API, and the zero-ETL data lake for custom reporting or integration with your existing analytics workflows.

What to do

• Monitor AI agent performance using the new metrics.
• Evaluate tool selection and utilization accuracy.
• Capture customer feedback through thumbs up/down ratings.

Source: AWS release notes

If you need further guidance on AWS, our experts are available at AWS@westloop.io. You may also reach us by submitting the Contact Us form.