Teams continue to burn through tens of thousands of tokens and agent calls as they refine exploration loops — and much of that spend, Google researchers argue, goes to paths that previously failed. Their answer is to let agents "dream" their way through those loops instead, learning from earlier attempts to guide future searches. They call it Dream-RSI, and in their tests it needed up to 162x fewer discovery-agent calls than SimpleTES, an existing discovery system.
The system is built on a lightweight orchestration layer that controls branching, parallel exploration and stopping decisions without changing the underlying coding agent. Exploration is “explicit and programmable” and the system evaluates and refines policies without repeatedly invoking costly online reruns.
Across several domains, Dream-RSI matched or improved discovery quality while substantially reducing compute costs.
“Because all execution outcomes are already saved in the tree, evaluating a new strategy requires only reading past records without rerunning the underlying discovery agent or evaluator,” the researchers explain.
The research comes days after Dario Amodei published his essay “We Must Pace the Frontier” on September 12. The Anthropic co-founder named recursive self-improvement, or RSI, as one of two developments that changed his mind about the need to slow AI development, because it could lead to AI that becomes too powerful for human control.
But this new experiment, based on benchmark results, highlights the gap between the existential threat Amodei and others warn about and the narrower, pragmatic uses for self-exploring agent loops.
Building "historical discovery trees"
RSI is all about effective exploration — where and when to branch, run in parallel, cut a line off — but managing that process can be problematic as discovery loops grow to thousands of cycles. Exploration is still “hand-written and frozen,” the researchers write on the project site, and fixed strategy doesn’t learn from accumulated experience, so builders keep paying for paths that previously failed.
“Poor exploration can waste substantial computation and time, severely limiting the efficiency and scalability of RSI,” they write. The bottleneck is “meta-level feedback” because it is delayed, expensive, and ultimately unnecessary. Builders are essentially reproducing feedback they’ve already paid for.
The team — 17 researchers across Google, Google DeepMind, the University of Maryland and the University of Virginia — argues that discovery should be represented as a structured “historical discovery tree” featuring every agent decision and resultant outcome. This gives the model the ability to choose what avenue to pursue next, without having to re-run anything.
“It walks the same recorded tree in a different order, and every outcome it asks for is already on disk,” the researchers write. The system can screen thousands of past discoveries without additional executions, and choose a winner for rollout. This essentially creates a “replay simulator” based on previously accumulated search history.
Importantly, the human-set exploration policy dictates where an agent searches, which attempts should run in parallel, and when exploration should stop.
Dream-RSI runs in three stages: "online explore" where policies guide an agent to build a discovery tree and log historical traces; a "construct replay simulator," where the tree and its traces are added to a reusable repository; and "dreaming-based policy improvement," where the agent essentially dreams up alternative policies.
The agent explores, appends its discovery tree as needed, rewrites policy, scores each version, then redeploys the best option. Every lap “adds a world”: The winning policies go back online to create a new discovery tree.
“That history has been sitting there all along; prior work just read it as something else — static textual context to prompt against, or training data to fine-tune weights on," the researchers write.
“History is the world to dream in,” the authors argue. “The discovery tree the agent already built is an exact simulator of the search space it reached.” The model improves through dreaming, and tests a new exploration policy by “imagining it” rather than repeatedly living through it.
“A simulator is anything that can answer ‘what would have happened if …’ without running the world,” they write.
How Dream-RSI fared on key benchmarks
Across a range of algorithmic, GPU-kernel and mathematical-optimization tasks in successive discovery rounds, the researchers found that Dream-RSI’s learned exploration policy adapted its compute use to generate stronger results with fewer agent calls than existing approaches.
In Lasso discovery tests — where agents are given a coding-and-optimization problem — Dream-RSI delivered a better quality-to-compute tradeoff than Recursive Fixed Exploration. RFE is when an AI agent explores a search space using a fixed orchestrator rather than adapting with time.
Using Gemini-3.1-Pro, Dream-RSI cut average runtime across six datasets from 3,587.1 to 2,931.0 milliseconds, and reduced discovery-agent calls from 550 to 317. With Gemini-3.7-Flash, it dropped average runtime from 2,516.7 milliseconds to 2,350.6 milliseconds, and reduced calls from 3,200 to 1,879. SimpleTES, the baseline behind the 162x figure, ran a different underlying model — gpt-oss-120b, at 51,200 generations.
The researchers evaluated Dream-RSI in GPU kernel engineering, where agents are tasked with discovering high-performance kernel implementations without sacrificing numerical correctness. This requires simultaneous reasoning around memory access, parallelization, algorithms, and hardware-specific optimization. That makes the benchmark suitable to determine whether Dream-RSI can generalize beyond pure mathematical discovery.
On the ConvDiv GPU-kernel task — which evaluates how efficiently hardware-level machine learning workloads run on a graphics card — Dream-RSI’s performance successively improved, cutting evaluated attempts from 110 to 50 before spending again, up to around 90, when progress plateaued.
On the GPU programming tasks VGG16 and LayerNorm, Dream-RSI reached comparable performance using 2.43x and 1.79x fewer generations. On ConvDiv and ConvMax, it achieved 2.09x and 1.44x higher performance under comparable budgets.
“Dreaming inside the replay simulator built from historical discovery trees buys immediate, low-cost off-policy feedback for evaluating and refining exploration policies, without repeating expensive online evaluations,” the researchers write. For teams already burning through agent calls, the gain comes from records they have paid to generate once — and the code is public.