Amazon CloudWatch Omni brings AI-powered observability to AI agents with dual IDE and web console

Amazon CloudWatch Omni is now generally available, giving teams building AI agents a unified place to trace, evaluate, and experiment across models and frameworks. The service lives both in your IDE (VS Code and Kiro) and in a standalone web console, sharing the same data so a developer's debug trace is the same trace an operator investigates. The IDE extension is free and works without an AWS account during development, making it easy to try out with a sample project or your own agent.

Two surfaces, one data model

Developers get a native VS Code and Kiro extension that surfaces traces as they run their agent. The Trace Explorer shows a structured timeline of every LLM call, tool invocation, and reasoning step, with drill-in access to inputs, outputs, token usage, and latency. You can compare two traces side by side in Compare mode or ask the built-in Ask Assistant to explain unexpected behavior.

Operators get a standalone web console accessible via SSO without the AWS Management Console. It includes the same Trace Explorer, plus a Session Explorer for multi-turn conversations, an Agent Topology view for system architecture, and Experiment capabilities to run test datasets against agent variants. The cloud connection is optional: you can use Omni entirely locally, then connect to AWS for production telemetry and team sharing.

Built-in evaluators close the quality gap

Traditional metrics like latency and error rate can't detect when an agent's response becomes less helpful or hallucinates. CloudWatch Omni includes 17 built-in evaluators scoring for coherence, helpfulness, faithfulness, routing correctness, and more. Trigger evaluations from any trace, get per-example scores and aggregates, and use the Playground to test prompt variations before deploying. With Prompt Management you can version prompts and roll back when a new version underperforms.

Supports what you already use

Omni works with the agent frameworks teams are already using: LangChain, LangGraph, CrewAI, OpenAI SDK, Vercel AI SDK, and Amazon Bedrock AgentCore in Python and TypeScript. Instrumentation uses open standards (OpenInference and ADOT) with OpenTelemetry, whether your agents run on Lambda, ECS, EKS, or other clouds. You can auto-instrument with Kiro or add manual instrumentation with provided code snippets.

Pricing and caveats

The IDE extension is free, and you only need AWS credentials if you use Bedrock models or API keys for providers like OpenAI or Anthropic. Telemetry shipped to AWS will incur standard CloudWatch costs for storage and data transfer. The initial IDE support is limited to VS Code and Kiro; other editors are not yet supported. While Omni uses open instrumentation standards, the management surface is tied to AWS, so teams already invested in other clouds or on-prem monitoring will need to evaluate the lock-in trade-off.