AWS's Infinity Research Runtime Puts Waiting AI Agents to Sleep; AWS Says 75,000 Fit in 8GB

AWS has open-sourced Infinity, a research runtime that suspends an AI agent every time it calls a tool and wakes it only when the result arrives. Released on October 23, 2026 under the Apache 2.0 licence, Infinity comes out of the Hydro project, a collaboration between AWS and UC Berkeley. Its pitch is simple: agents spend most of their time waiting, so a waiting agent shouldn’t hold a process, a thread or much memory. In AWS’s own benchmark, which hasn’t been independently reproduced, about 75,000 idle agents fit in 8GB, the memory of a Raspberry Pi 5.

The problem with blocking agent loops

Most agent frameworks run a loop of blocking steps: call the model, call a tool, wait for the tool, repeat. Shadaj Laddad, research lead for the Hydro team at AWS, describes the cost in the announcement. A coding agent may start a 20-minute build, a triage agent may wait hours for the next pull request, and an assistant may wait overnight for an approval. Throughout, the agent holds a process and its memory, and it cannot react to anything else that happens.

Infinity borrows an idea from programming languages. Rust, JavaScript and Python suspend async functions at each await and save only the state needed to resume. AWS’s observation is that an agent works the same way: when it calls a tool, its conversation history already contains everything needed to continue.

How a slice works

Infinity runs each agent as a series of short “slices”. In a slice, the runtime loads the agent’s history from storage, runs one model completion, saves the output and sends off at most one tool call. Then the slice ends. Tools don’t return results directly. They start their work, return at once, and later post the result to the agent’s input queue, which starts a new slice. User messages arrive through the same queue.

Agents are organised much like an actor system. Each conversation thread has its own queue, threads share no mutable state, messages for one thread are processed in order, and different threads run in parallel. A thread with nothing queued has no running task at all.

AWS’s benchmark ran 80,000 agents for 20 turns each, 3.2 million completions in total, on a single thread of an AMD EPYC processor with a simulated model and in-memory storage. Memory grew by about 100KB per idle agent, mostly conversation history.

Agents that cost nothing while they wait

Because no slice waits on anything open-ended and all state lives in storage, the same code can run on serverless platforms. AWS’s reference deployment uses an SQS FIFO queue with one message group per thread, a Lambda function that runs one step per delivery and exits, Aurora DSQL for conversation histories and DynamoDB for tracking which messages have been handled. Timers use SQS delivery delays and EventBridge Scheduler, and the repository includes CDK constructs to set it all up. An agent waiting on a three-day CI pipeline uses no compute until the pipeline reports back.

Queues can deliver a message twice, so each input carries a stable ID and repeats are skipped. Model output is committed only when a turn finishes, and a tool call is saved before it is sent, so its result can never arrive first.

Fake tool calls for real events

Model APIs expect every tool call to get exactly one result, which doesn’t fit events such as “a new pull request was opened” that keep arriving. Infinity’s answer is a synthetic tool call. When an event comes in, the runtime adds a call the model never made, named receive_event__injected, with the event as its result and arguments pointing back to the original subscription. The model is never offered that tool name, and the runtime rejects it if the model tries to call it.

The same mechanism handles sub-agents. A built-in spawn_thread tool starts a child thread, and the child’s reports come back to the parent as events on that call. By default, each incoming event is handled in a short-lived child thread, so a busy subscription doesn’t fill the parent’s context window.

Early, and openly unfinished

AWS calls Infinity an early research release and lists open questions. One is how well models reason about interleaved events when they are mostly trained on synchronous tool calls. Others are how to schedule threads by priority, how to compact long trees of agents, and how to verify that events from the internet are genuine. The Rust code is in the hydro-project/infinity repository on GitHub, with a quickstart for building a first agent.