T-Mobile's first attempt to give its AI customer service agents enough context backfired. The agents had access to a large body of support documentation, but conflicting instructions could send them down the wrong path for a customer's request, according to CIO Jeff Simon at VentureBeat's Agentic Infrastructure Stack Summit in October. A subsequent knowledge graph proved too slow and difficult for agents to navigate.  

"We said, 'Oh, we've finally got a big context window model. Let's load all of our customer support documentation into that agent, and it's gonna be the expert, right? It's gonna know everything,'" Simon said. "That didn't work at all."

In VentureBeat Intelligence's October VB Pulse survey on agentic context layers, 78% of the 97 respondents whose organizations have AI agents working on company data traced at least one confident but wrong answer to missing or inconsistent business context in the past six months, which the survey defines as a context failure.

Among the 76 respondents who reported a context failure, 42% said their most significant case resulted in incorrect information reaching an outside party or official report, or in a decision or action being reversed.

T-Mobile has been building agents that answer customers over chat, text and voice since 2024. It now relies on a simple lookup table that maps common ways customers phrase questions to the paths agents should follow. 

Conflicting documents and a slow graph led T-Mobile to a lookup table

T-Mobile has focused on orchestrating its customer-service agents rather than training its own models, Simon said. As the models have improved, T-Mobile has added more agents, he said, because the models have gotten better at coordinating multiple agents and tool calls. Before a request reaches the model, the company removes content that may not be relevant, a step Simon called one of its most important learnings.

The conflict in the documentation is what sank the first approach. Simon said the support documentation of a company the size of T-Mobile contains conflicting guidance, and a customer's words can also map to more than one process. A customer who asks for a new device, for example, may want a new phone on an existing line or an additional line.

"When you load up the context window really large, the AI is gonna be forced to make a decision between the paths and the documentation, and it may start answering from the wrong ones," Simon said.

T-Mobile's next attempt was a knowledge graph.

"The knowledge graph didn't work largely because the agents had to search across this big graph, and it took time, and we don't have a lot of time," Simon said.

Speed was not the only problem, because customers ask the same question in many different ways. When agents misinterpreted a request, they could take the wrong route through the graph and get lost, even when T-Mobile tried to guide them.

T-Mobile replaced the graph with a simple cache, stored in a key-value database, that works as a lookup table. The table maps the ways customers commonly phrase questions to where an agent should go next, and the team updates it as customers ask in new ways. An orchestrator agent then confirms which path a customer wants and sends the request only to the agent holding context for that path. The table is built for agents rather than people.

"It is actually very basic, but it's fast and responsive, it scales really well, and so far the performance from our agents is good," Simon said.

T-Mobile's agents can also take limited action on customer accounts. Simon said customers can use an agent to purchase an international data pass or add other services, with checks to confirm that the customer intended to make the purchase. 

T-Mobile also runs a small language model alongside its production model on every chat and voice session. It works as an LLM judge, flagging sessions where an answer went wrong or a customer is getting angry so that humans can review them, since the volume is too high for people to review all sessions or even a large sample.

Running a second model on every interaction adds cost, Simon acknowledged, but T-Mobile considers it worthwhile because the team cannot anticipate every unusual way a customer might phrase a request.

The judge flagged hold music playing on many calls, a surprise because T-Mobile does not play hold music. Engineers spent a couple of weeks examining the calling and interactive voice response systems before they found the cause in the customer-facing model. Simon said the model was generating hold music when it paused to look something up or run a tool call.

"It was completely invented," Simon said.

T-Mobile now tells the model in its prompts not to hallucinate hold music, a problem the team had never imagined facing.

Wrong answers sometimes reach customers and board reports

The VB Pulse Agentic Context Layers survey puts numbers on what is at stake when an agent gets business context wrong. Among the 76 respondents who reported at least one context failure, 58% said their most significant case was caught internally with no external impact. Another 21% said a customer, partner or supplier received incorrect information. Twelve percent said an incorrect figure reached a financial, regulatory or board report, and 9% said a decision or action had to be reversed.

Respondents could choose more than one way they would normally find out about a confident but wrong answer. Automated checks that compare agent output against a trusted source were the most common at 57%, followed by subject-matter expert sample review at 55% and dashboard or monitoring alerts at 52%. A customer, partner or supplier noticing was named by 19%.

Automated checks went together with root-cause tracking, since all 55 respondents who named them also track whether a wrong answer came from business context or from the model. Of the 42 who did not name them, 10 (24%) do not.

Adobe and Uber use knowledge graphs. T-Mobile’s agents got lost in one

Adobe frames the choice between more and less context differently. At the Agentic Infrastructure Stack Summit, Anjul Bhambhri, senior vice president of engineering at Adobe, described Adobe CX Enterprise Coworker, the company's marketing agent, as aiming for precise context. The amount depends on the task.

At the narrow end, an email marketer in Europe needs the email requirements, the EU's consent policies and the company's own rules on whom to contact.

"If you feed both the EU and the US consent policies, now there are chances of errors," Bhambhri said.

At the other end, refining a holiday campaign calls for years of history.

"There's a lot more context that has to be given because if that is not given, they may get a very generic response," Bhambhri said.

In one case Bhambhri said historical data showed that free shipping converted better than a 20% discount for loyal customers during a Black Friday sale.

Adobe narrows context before the model sees a prompt, Bhambhri said. The process starts with seed context, which is fixed for a company and covers what every user needs, such as brand guidelines. A prefetching step then draws on keywords, semantic indexes and knowledge graphs. The graphs track how marketing artifacts connect, what was used when and what results followed. A merging and ranking step then decides how to fill the context window. Natural language understanding and a cheaper model handle this work ahead of the main model.

Uber's experience with a knowledge graph was different from T-Mobile's. In an August 27 engineering blog post, the company said an agent grounded in its AI Context Graph found a data table in 38 seconds, while the same model without the graph spent more than 20 minutes and wrongly concluded the data could not be queried. As VentureBeat reported Thursday, Uber's engineering agents use the graph to locate relevant systems and retrieve documents or code as needed. T-Mobile's agents, by contrast, had to navigate theirs during live customer interactions.

Few respondents rely mainly on large context windows

The VB Pulse survey asked respondents to name the primary source of business context, and 2% said their agents rely on the model's general knowledge with no enterprise context layer.

Retrieval over documents or a vector index led at 32%, followed by direct queries to live systems at 25% and a governed semantic layer or ontology at 16%. Another 14% said the mix varies by use case. Just 10% named loading large inputs directly into the model's context window as their agents' primary source of business context.

For teams building agents on company data, the questions are practical: Which documents in an agent's context conflict? Would the team find out if the agent answered from the wrong one? Who owns the lookup table, index or graph that decides what the agent reads? How long can a customer wait for a retrieval step?