Every agent that searches a document corpus has to decide how many times to search before it answers a question.
Most agentic search tools force a choice between two failure modes. Search that stops too early misses questions that require multiple hops of reasoning across documents. Search that never stops early runs the same number of steps on every query, including the simple ones that never needed more than one.
Databricks introduced Instructed Retriever in January, an architecture that Databricks says outperformed traditional RAG by up to 70% on complex, instruction-heavy enterprise questions, using metadata reasoning to do it. The system was built for a curated workspace, where teams defined a specific, smaller set of documents and tables for the retriever to search. That scope held until Databricks connected the same large language models to an entire enterprise workspace instead of a curated one.
In research published this week, Databricks introduced Adaptive Instructed-Retriever, a retrieval model built to decide how many steps a search needs before it runs. The difference comes down to when the model decides to stop. Databricks says the model matches the quality of Claude Sonnet 5, GPT-5.6 Luna and DeepSeek-V4-Flash while answering more than twice as fast, an average of 5.8 seconds. In one benchmark example from the research, it matched Sonnet's two step search on a customer account question while returning 50% higher recall. Those benchmark figures come from Databricks' own testing and have not been independently verified.
"The initial Instructed Retriever work, where the workflow was pretty fixed, was more similar to the query plan because we fixed the structure of how the agent calls the retrieval system," Michael Bendersky, research director at Databricks, told VentureBeat. "In this particular work, we move away from this to a more adaptive approach, where the agent adapts to the question and changes the plan based on the question."
Query plan vs. adaptive search
A traditional database query plan is the fixed set of steps a database follows to execute a query, such as which index to scan or what order to join tables. The same query produces the same plan every time. Bendersky used that comparison directly.
"The query plan setup was deterministic and rule based," he said.
Adaptive Instructed-Retriever shares one goal with a query plan and breaks from it on another. "It's similar to the query plan in the sense that we try to make the agent as fast as possible," Bendersky said. The difference is in how that speed gets decided. "It's different because the flow here is actually non-deterministic, meaning an agent can take one path for one query and a very different path for a different query," he said.
The clearest example of where that flexibility matters is what Bendersky calls multi hop questions, where answering requires more than one round of retrieval to assemble an answer. A question about revenue figures that live across two documents is one example. Answering it means retrieving the first document, pulling a pointer from it, then issuing a second search based on what that document contained. A single search pass will not find it, and a fixed plan finds it only by running every query through the same number of steps regardless of whether it needed them.
Two search modes, and a penalty for extra steps
The research describes two retrieval modes and a training method that decides when to use each.
Parallel thinking. The system issues several rewritten versions of a query at once, each sent by a separate thread, with the results merged at the agent level.
Sequential thinking. When an initial set of retrieved documents contains a clue but not the answer, the model uses that information to issue a second round of queries, repeating the process up to a fixed step limit.
The step penalty. Databricks trained the model with online reinforcement learning using a technique it calls CISPO, short for Clipped Importance Sampling Policy Optimization. "We train the agent to search more, but only when needed," Bendersky said. "We get a penalty that balances the time spent searching with the reward of finding the right documents."
Fewer steps than Claude, GPT-5.6 Luna and DeepSeek — not just faster
The comparison in the research is not against other retrieval products. It is against frontier models working the same search task. By adjusting the strength of the step penalty during training, the researchers produced a family of checkpoints instead of one fixed model, each landing at a different point on a quality-latency curve. Every checkpoint on that curve outperforms Claude Sonnet 5, GPT-5.6 Luna and DeepSeek-V4-Flash across the full range of retrieval budgets tested, according to the research.
The research includes another head to head example. Asked whether a company reported restructuring costs on a specific income statement line, all three models reached a perfect recall score. Adaptive Instructed-Retriever got there in two search steps, one fewer than Claude Sonnet 5 and two fewer than GPT-5.6 Luna, which searched for speculative phrases such as "there were no such costs" before confirming the absence.
Databricks frames the results as evidence that a small, specialized model can match frontier accuracy on a narrow search task without needing a larger general purpose model to do it.
Faster search doesn't fix bad context
The timing lines up with a harder problem enterprises are still working through. VB Pulse's July 2026 survey of 101 qualified enterprises found 68% had traced a confident but wrong AI agent answer to missing or inconsistent business context in the past six months, up from 57% in June. Retrieval over documents remains the most common way enterprises supply that context, the primary source for 31% of respondents.
Faster retrieval does not close that gap on its own. Response correctness is still the top metric enterprises grade agents on, even as buying criteria shift toward access control and ingestion ease over retrieval accuracy. How many steps a system takes to reach an answer is one question. Whether that answer rests on consistently defined business context is another, separate one.