Skip to main content

Session Management & Streaming

Who this is for

Developers designing session behavior, streaming UX, and memory isolation for a conversational agent. If you haven't deployed a prototype yet, start with the Quick Start or Harness Quick Start first.

Conversational agents need stateful, long-lived sessions, not the request/response model of traditional APIs. AgentCore handles this at the platform level.

How Sessions Work​

CapabilityWhat AgentCore Provides
Session isolationEach session runs in a dedicated microVM. No shared state between users.
Session durationCompute lifecycle lasts up to 8 hours. Long enough for complex multi-step workflows.
Session persistenceDurable state stored in AgentCore Memory survives client disconnects. In-flight microVM state does not persist across a cold restart.
Session ID trackingPass a runtimeSessionId on each invoke. Same ID = same conversation.
Scale-to-zeroIdle sessions terminate (default 15 min). You pay only for active time.
Architecture guidance

Session duration, idle timeout defaults, and reconnect behavior are architectural properties that may vary by configuration and SDK version. Consult the AgentCore service quotas for current limits and configurable values before designing against specific numbers.

Multi-Turn Conversations​

AgentCore Memory keeps your agent coherent across turns without manual context plumbing:

  • Short-term memory (session) — automatic when memory is configured. Every message in the current session is available to the agent. No prompt stuffing required.
  • Long-term memory (cross-session) — opt-in. Store user preferences, past decisions, and facts that should persist across sessions. Namespace per user for isolation (see isolation design note below).
  • Summarization — when a SUMMARIZATION strategy is configured, AgentCore Memory can produce summaries of older turns so the agent retains the gist without exhausting the context window. This is a configured strategy, not automatic behavior on every deployment.

Handling Interruptions and Resume​

Real users close laptops, switch devices, and come back hours later. AgentCore is designed to support reconnection:

  1. Durable state in AgentCore Memory — facts and summaries stored in memory survive client disconnects and cold restarts because they are persisted in the memory service, not only in the microVM.
  2. Resume with the same session ID — passing the same runtimeSessionId on a new invoke allows the agent to retrieve prior context from memory and continue the conversation.
  3. Timeout behavior — after the idle timeout (configurable, consult the AgentCore service quotas for current defaults and limits), the microVM terminates. State held only in the microVM process is not recoverable; state committed to AgentCore Memory is.

Cross-Channel Continuity​

Users may start on Slack, switch to a web app, then follow up on mobile. Because sessions are server-side and identified by runtimeSessionId + user identity:

  • Same user + same session ID = continuation regardless of channel.
  • AgentCore Identity links the user across channels via their IdP token, not a device cookie.

Context Window Management​

Long conversations consume context windows quickly. As multi-turn conversations with tool calls grow, raw message history can exhaust the model's context window, degrading answer quality, increasing cost, and eventually hitting hard model limits. Without management, agents degrade silently: they don't crash, they just start giving worse answers.

Short-term memory with automatic summarization — recent turns are kept verbatim; older turns are summarized into concise recaps that preserve key facts and decisions.

Semantic retrieval from long-term memory — when the agent needs a fact from many turns ago, AgentCore Memory embeds the current query and retrieves the most relevant past events by semantic similarity. Only the relevant snippets are injected into context.

Namespace scoping per user — memory is isolated by user/actor ID. One user's long conversation doesn't pollute another's context.

PracticeWhy
Use short-term + long-term memoryShort-term for the active thread; long-term for cross-session facts
Set a summarization windowSummarize after N turns to keep active context within model limits
Retrieve, don't replayFetch relevant history on demand instead of replaying all messages
Separate system prompt from historyPin critical instructions outside the sliding window
Monitor token usage per conversationTrack via CloudWatch to catch runaway conversations early

Streaming & Real-Time UX​

Conversational agents live or die on perceived responsiveness. AgentCore streams tokens to the client as they're generated, with no buffering.

CapabilityDetail
Token-by-token streamingSSE from the /invocations endpoint for progressive response delivery.
Tool-activity feedbackDuring tool calls, emit status events so the UI shows "Looking up account…"
Partial response displayStream the answer while tools run in parallel behind the scenes.
WebSocket supportBidirectional streaming via WebSocket for real-time UX — verify endpoint availability and configuration in the AgentCore WebSocket protocol contract.

Without streaming, users wait on a blank screen while the agent reasons and calls tools. Perceived latency is a leading cause of enterprise chat-agent abandonment.

Multi-Tenant Isolation​

Enterprise deployments serve multiple teams, business units, or customers from shared infrastructure. AgentCore provides tenant isolation at the platform level.

LayerIsolation Mechanism
ComputeDedicated microVM per session. No shared OS processes between tenants.
MemoryNamespace-per-user. One tenant cannot retrieve another's memory events.
Gateway/ToolsAuth propagation ensures each tool call is scoped to the user's token.
ObservabilityTraces and logs carry user context; RBAC on CloudWatch log groups.
Data at restEncryption with customer-managed keys (CMK) supported.
Why this matters

Without platform-level isolation, teams build custom tenant partitioning into every layer: DynamoDB partition keys, S3 prefix policies, Lambda authorizers, log filtering. AgentCore does this once, correctly, at the infrastructure level.

Memory Namespace and Isolation Design​

AgentCore Memory uses actor_id to namespace memory events per user. The platform enforces that namespace boundary — but it cannot enforce that your application supplies the right actor_id value in the first place.

When retrieving memory records, use namespace to filter by exact prefix match, or namespacePath for hierarchical retrieval that returns all records under a parent namespace. If your namespace design nests scopes (for example, tenant under organization), namespacePath is the parameter that returns everything under a given parent level; namespace alone only matches records whose namespace starts with the exact string you provide. See RetrieveMemoryRecords for the full parameter reference.

Isolation only works when identity is correct. If your code passes a shared constant (e.g., "default-user"), all users who share that value read and write into the same memory namespace. The result is cross-user data leakage with no platform-level warning.

Design checklist before enabling long-term memory in a multi-user deployment:

  1. actor_id is derived from a validated identity claim — not a client-supplied header, not a default fallback, not a constant.
  2. session_id is unique per conversation — not reused across users or shared across channels without intent.
  3. If you use AgentCore Identity, the validated token claim is the source of truth for both values.
  4. Test isolation explicitly: create two users, store a fact as User A, query as User B, and confirm User B cannot retrieve User A's fact.

Data Governance for Memory​

Conversational memory stores information about users across sessions. Before enabling it in production, address these governance considerations:

Data governance checklist
  • Tenant boundaries — verify that your actor_id and namespace design prevent cross-tenant access (see above).
  • Retention — configure eventExpiryDuration to the shortest period your product requires. Retaining data longer than necessary increases privacy risk.
  • Deletion and forgetting — plan for user deletion requests (e.g., GDPR right to erasure). Confirm your deletion workflow against the AgentCore Memory documentation.
  • Sensitive data minimization — avoid storing raw PII, credentials, or regulated data (PHI, PCI) as memory events. Extract only the facts the agent needs to function.
  • User consent — if your deployment is user-facing, disclose that the agent retains information across sessions and give users a way to clear their memory.

Further Reading​