Generative AI observability now generally available for Amazon CloudWatch

Amazon CloudWatch Generative AI Observability

Amazon CloudWatch has introduced generative AI observability, providing comprehensive monitoring for all components of AI applications and workloads, including agents deployed with Amazon Bedrock AgentCore. This feature offers complete observability across AgentCore's Built-in Tools, Gateways, Memory, and Identity capabilities. Developers can now monitor latency, token usage, errors, and performance across all components of their AI workloads, from model invocations to agent operations.

This feature is compatible with popular generative AI orchestration frameworks such as Strands Agents, LangChain, and LangGraph. It enables developers to analyze telemetry data across components of a generative AI application, including code execution patterns, API transformation success rates, memory storage and retrieval patterns, and secure agent behavior.

Developers can monitor their entire agent fleet through the "AgentCore" section in the CloudWatch console, which integrates with other CloudWatch capabilities including Application Signals, Alarms, Sensitive Data Protection, and Logs Insights.

What to do

• Monitor your AI workloads using the new AgentCore section in the CloudWatch console.
• Utilize end-to-end prompt tracing, curated metrics, and logs to identify issues quickly.
• Integrate with popular generative AI orchestration frameworks for flexibility.

This feature is available in multiple regions including US East (N. Virginia), US East (Ohio), US West (Oregon), Europe (Frankfurt), Europe (Ireland), Asia Pacific (Mumbai), Asia Pacific (Tokyo), Asia Pacific (Singapore), and Asia Pacific (Sydney).

Source: AWS release notes