An enterprise AI agent may need to find relevant business information, revisit earlier decisions and inspect operational data, often through multiple retrieval steps, before it can complete a single task. The question for enterprise leaders is how to supply that context reliably without letting infrastructure costs grow faster than the value agents deliver.

VentureBeat’s July context-layer survey found 68% reported context-related agent errors; 32% had governed context layers in production (101 respondents). That gap frames the challenge for OpenSearch: making information retrievable is only part of making it trustworthy.

Tia White, general manager of OpenSearch at AWS, is among the keynote speakers at OpenSearchCon North America, September 22 to 24 in San Jose, alongside Apple's Atri Sharma and Abhishek Singh; IBM's Pushkala Pattabhiraman; CERN's Sokratis Papadopoulos; Uber's Mingmin Chen; and OpenSearch Software Foundation Executive Director Bianca Lewis.

I’ll be there, too. If you’re attending and working through questions about retrieval, agent memory or the economics of production AI, I’d welcome the chance to connect.

My recent conversation with White provides a useful starting point. She described an AWS strategy that brings multiple retrieval methods together, addresses the unpredictable demand agents create and expands the role of search infrastructure in managing context.

Retrieval is becoming a broader infrastructure decision

Amazon OpenSearch Service combines capabilities that enterprises often evaluate separately: full-text search, vector retrieval, hybrid search and operational analytics. 

Hybrid search is particularly relevant to enterprise AI. Semantic retrieval helps an application find information with a similar meaning, while lexical matching preserves the precision needed for product identifiers, account numbers and technical terminology. A useful enterprise answer can depend on both.

In our conversation, White emphasized the opportunity for customers to extend an existing OpenSearch deployment into AI workloads. For organizations already operating the service, that could reduce the need to introduce another database, security configuration and operational workflow.

My view is that this changes the evaluation criteria. The question becomes whether a retrieval platform can support the application’s complete context requirements at an acceptable operating cost. Vector performance is one part of that decision. Relevance, permissions, data freshness and integration effort matter alongside it.

The distinction between the open-source project and AWS’s managed service also matters. OpenSearch moved to the OpenSearch Software Foundation under the Linux Foundation in 2024. Amazon OpenSearch Service is AWS’s managed offering built around the technology; its commercial roadmap should be assessed separately from the community project’s governance and development.

Agents change the economics of search

An agent’s demand for retrieval can vary substantially within a single workflow. It may search, inspect the results, reformulate a question and retrieve again. Across many agents, that pattern makes capacity planning harder.

AWS’s OpenSearch Serverless service, generally available since May, targets that problem. It separates compute and storage and, according to AWS, provisions resources in seconds with autoscaling up to 20 times faster than its predecessor. AWS also claims savings of up to 60% compared with provisioning OpenSearch clusters for peak demand. Those are vendor claims tied to a particular comparison, rather than a forecast for every deployment.

The practical opportunity is to reduce how much idle capacity a team must maintain to accommodate bursts. Scale-to-zero also needs careful interpretation: AWS documents it for eligible next-generation collections in collection groups. Teams should verify their configuration and measure the behavior against their application’s response-time requirements. AWS’s scale-to-zero documentation explains those conditions.

For enterprise buyers, the useful measure is total cost per successfully completed agent task, not cost per token. A low query price can lose its appeal if poor retrieval causes repeated searches, larger model inputs or failed workflows.

The same agents create an observability bill

Agents consume context while generating logs, traces and records of tool activity. That creates another infrastructure challenge: retaining enough evidence to understand what happened when an agent fails.

AWS’s optimized log analytics engine combines columnar analytics with full-text search. Columnar processing supports aggregation and trend analysis; full-text search helps investigators locate a particular error or event. AWS advertises up to four times better price-performance and twice the analytical-query speed for the optimized engine.

This could help teams move from identifying a pattern to investigating the underlying events with less data movement. The business value depends on whether that reduces investigation effort or allows useful history to be retained economically.

There is an important architectural boundary, however. AWS recommends the optimized engine for log analytics and the general-purpose engine for workloads requiring capabilities such as vector or semantic search. A common service does not mean every workload runs through the same engine configuration.

Enterprise architects should evaluate the operational benefits of consolidation while preserving the workload choices each application needs. 

Agent memory and institutional memory are not the same thing

White identified memory, query planning, reasoning and evaluation as areas where retrieval infrastructure can contribute more value.

Memory is already moving beyond positioning. OpenSearch 3.5 introduced persistent agent conversation memory, and AWS announced support for that version in Amazon OpenSearch Service in March. The capability captures conversation context and tool reasoning across interactions.

The larger enterprise challenge is deciding what deserves to become durable agent memory versus institutional memory that is key intellectual property.

Conversation history records what an agent or user said. Institutional context includes authoritative policies, permissions, business definitions and approved decisions. Storing both does not automatically establish which source should take precedence.

In my view, the next test for retrieval platforms is how effectively they help applications manage those distinctions. Teams need ways to preserve provenance, apply corrections, expire stale information and prevent an agent’s mistaken output from returning later as trusted evidence.

At OpenSearchCon, I’ll be looking for implementation examples that show how practitioners are handling those problems, alongside their relevance and performance results.

How to test this against your own stack

The decision should begin with one representative workflow.

Business and application owners should define what successful agent task completion looks like and assemble realistic retrieval questions, including exact identifiers, ambiguous requests and restricted information. Platform teams should test bursts and idle periods while measuring response times and total operating costs. Data and security owners should establish which sources an agent may retrieve, what it may remember and how corrections or deletions reach that memory.

Those results will determine whether expanding OpenSearch simplifies the architecture and improves outcomes.

My main question heading into OpenSearchCon is how well the ecosystem can connect its expanding technical capabilities to trustworthy application behavior. The strongest evidence will come from production teams showing that their agents retrieve the right context, respect its boundaries and complete useful work at a sustainable cost.