Amazon Bedrock AgentCore Observability Solves Operational Challenges

The AWS machine learning blog “Optimizing production agents with Amazon Bedrock AgentCore Observability” discusses the challenges faced by AI agents when transitioning from prototypes to production. According to the article, the challenge in the prototype stage is “whether it works,” but in the production stage, the challenge shifts to “whether it works quickly and efficiently.” This is the second part of the series, with Part 1 focusing on debugging “broken agents” with issues such as infinite loops or tool invocation errors, while this article addresses the different challenge of “agents that work correctly but have poor performance.”

The author identifies response latency and unbounded memory increase as “the most common operational issues.” These problems are difficult to detect because they do not trigger error alerts, but over time, they erode user trust and increase costs. This is the core argument of the article, which explains how to use Amazon Bedrock AgentCore Observability (a feature of Amazon Bedrock AgentCore) and Amazon CloudWatch to identify performance bottlenecks in the entire execution path of an agent and diagnose memory issues in long-running sessions.

The prerequisites for this process include having access to Amazon Bedrock AgentCore, enabling CloudWatch Transaction Search, and having a deployed agent. The detailed setup instructions are provided in Part 1 (linked in the article as “Debugging production agents with Amazon Bedrock AgentCore Observability”).

Symptoms and CloudWatch-Based Identification of Performance Bottlenecks

The article discusses “Scenario 3: Performance bottlenecks” in detail, where the agent works correctly but responds too slowly. This is a situation where sub-second responses are expected, but delays of several seconds occur, and while the task itself is successful, the latency becomes impractical for interactive use cases. The article states that “slow is subjective,” highlighting that delays acceptable in batch processing agents may not be acceptable in customer service chatbots.

The symptoms of performance degradation are often gradual, the article explains. Initially, the response time may be around 2 seconds, which is acceptable, but as features are added, tools are integrated, and memory accumulates, the response time worsens to 5 seconds, 10 seconds, and eventually becomes unusable. During this period, the P95 response time exceeds the threshold, causing users to abandon sessions, while the error rate remains low. Figure 1 in the article shows three requests with consistently high latency (7.5-8.2 seconds average span latency), which is indicative of “systemic performance bottlenecks” rather than sporadic slowness.

To find the bottlenecks, the article first shows how to query agent invocations that exceed the performance budget in CloudWatch:

fields @timestamp, RequestId, Latency | filter Operation like /InvokeAgent/ | filter Latency > 3000 | sort Latency desc | limit 50

This query returns agent invocations that exceed 3 seconds (Latency > 3000) in descending order of latency. The threshold can be adjusted according to specific requirements. By selecting a representative high-latency request and recording its RequestId, the next query can analyze the entire timeline of the request:

fields @timestamp, Operation, Duration, SpanName | filter RequestId = "" | sort @timestamp asc

This query shows the execution order of operations within the request and the time each operation took. The article advises looking for operations that consume a lot of time (Duration is large) in this timeline.

Diagnosing Memory Issues in Long-Running Sessions

As stated at the beginning of the article, Amazon CloudWatch is also used to diagnose memory issues in production agents. Along with response latency, “unbounded memory increase” is listed as one of the most common problems in production, particularly in long-running sessions.

However, the provided source information is limited to the introduction and the first scenario (performance bottlenecks), and the article is cut off before detailing the specific CloudWatch queries or steps for memory diagnosis. For detailed steps on diagnosing memory issues, one would need to refer to the continuation of the article or the official AgentCore Observability documentation (docs.aws.amazon.com/bedrock-agentcore/latest/devguide/observability.html).

The article guides readers to check AgentCore Evaluations and AgentCore Insights for additional information on performance optimization and best practices. These are also features of Amazon Bedrock AgentCore, but their specific functionalities and usage are not explained in this article. Instead, readers are directed to the official documentation for details.

When starting to operate production agents, the first step, as indicated by the article, would be to open the AgentCore Observability official documentation (docs.aws.amazon.com/bedrock-agentcore/latest/devguide/observability.html), enable CloudWatch Transaction Search, and deploy the agent.

Summary

  • Using CloudWatch queries like Latency > 3000 to filter agent invocations and sorting them by latency in descending order allows for the identification of specific RequestIds that have “systemic bottlenecks” from thousands of logs, which can then be prioritized for investigation.
  • By applying a timeline analysis query (Operation, Duration, SpanName) to a high-latency RequestId, it’s possible to identify which operations (tool invocations, model inferences, etc.) within the request are consuming time, enabling data-driven optimization rather than blind tuning.
  • Introducing threshold-based monitoring, such as P95 response time, allows for the detection of “degradation that causes users to abandon sessions” before error alerts are triggered, which may not be captured by error rates alone.
  • Combining the performance diagnosis methods using AgentCore Observability and CloudWatch with the debugging methods for infinite loops and tool invocation errors covered in Part 1 enables the construction of a continuous production monitoring system that can handle both “broken agents” and “agents that work but are slow.”