Cost overruns stalled AI projects more than three times as often as computing-capacity shortages among respondents to VentureBeat’s October survey. 

In the past 12 months, 38% said their organization delayed, scaled back, or canceled a planned AI project because it cost more than expected, compared with 12% who cited insufficient computing capacity. The October VB Pulse survey on AI infrastructure and compute covers 156 qualified respondents working on AI at, or consulting for, organizations with at least 100 employees. 

Investors, meanwhile, are worried about the pace of future data-center construction. On Nov. 5, technology stocks fell sharply amid concerns that the midterm election results could slow AI data-center construction, Fortune reported. VentureBeat's survey was fielded in October and asked about project delays over the preceding year, not expectations for future capacity.

Despite cost overruns, 79% of respondents expect their organization’s AI budget to grow over the next year. Yet only 26% of all respondents say their organization judges AI spending by a measurable financial return it could defend to its finance team. Among respondents expecting a budget increase, the figure is 27%.

Companies need to be able to change their AI systems as they learn what each workload costs. A prototype’s model may become too expensive at scale, while repeated agent queries can drive up retrieval costs. Open-source code and open interfaces can help companies change models and providers without rebuilding every integration, and commercial vendors can support those systems in production. The test is whether those choices improve the cost and quality of completed work.

How Uber cut the cost of feature-flag cleanup by 90%

Uber VP and Distinguished Engineer Uday Kiran Medisetty described a practical example at VentureBeat’s Agentic Infrastructure Stack Summit in October. The company benchmarks managed agents against actual tasks and checks quality indicators such as reverts and incidents. For feature-flag cleanup, he reported a 90% reduction in cost per cleanup after benchmarking and selecting an appropriate model.

Uber’s published account also describes measuring cost by outcomes such as a completed review or cleanup. The result Medisetty reported is specific to Uber’s task. Other companies can use the same approach: Evaluate model choice against real work, then measure cost alongside quality.

The financial evidence behind AI spending

The survey also asked about returns on AI infrastructure. Among the 91 respondents whose organizations track those returns rigorously or partially, 34% said their finance team had reviewed and approved a return figure. That represents about 20% of all 156 survey respondents.

Industry groups are trying to close that measurement gap. J.R. Storment, executive director of the Tokenomics Foundation, a Linux Foundation group developing open standards for measuring AI costs and returns, described the need for a consistent record connecting technology activity to business value. “We’re like the ledger,” he told me, distinguishing that work from projects connecting agents to tools and models. 

The FinOps Foundation, which Storment also leads, already publishes FOCUS, the FinOps Open Cost and Usage Specification, a common format for billing data. Storment described work to connect cost and usage with telemetry, outcomes and a classification of value. That connection would help explain what a workflow accomplished for the money spent.

Switching AI models takes more than open weights

Better measurement matters because many companies are still changing their infrastructure. In the October survey, 56% of respondents said their organizations plan to adopt a new, additional or replacement AI compute platform within the next 12 months. A platform decision made during experimentation may need to change as deployment expands.

Open-weight models are part of that choice. Seventeen percent of respondents say their organization runs open-weight models for production workloads, while 33% use them only for internal or experimental work. Publicly released weights do not necessarily mean an open-source license, and those figures do not establish that an open-weight model will be cheaper for a given task.

Self-managed open-source infrastructure is less common: 7% of respondents who named a listed platform say their organization uses a self-managed open-source stack in production. Just 2% of all respondents named one as their primary AI compute platform. Those platform-level answers do not inventory the open-source dependencies inside managed services or packaged software.

An open interface, an open-source implementation and publicly released model weights provide different kinds of flexibility. Buyers should check licensing, compatibility, and support separately rather than treating openness as a single attribute.

One way companies can switch between models is through an AI gateway. Varun Talwar, Tetrate’s co-founder and CTO, described it as software that routes requests and enforces policies across external model APIs, internally hosted models, tools and security services. Tetrate builds on Agent Router, formerly Envoy AI Gateway, which joined the Agentic AI Foundation in September. That traffic layer serves a different purpose from Kubernetes workload management and inference projects such as vLLM and llm-d.

Talwar said traffic handling remains open source, while Tetrate sells management capabilities including multitenancy, administration and audit. Buyers should examine that boundary, the effort required to change suppliers and whether automated routing delivers savings without sacrificing quality. Talwar recommends evaluating workload telemetry, cost and quality before making automated model selection the default, including the expense of retries and human review.

Mazin Gilbert, executive director of the Agentic AI Foundation, described competing companies working on protocols, gateways, skills and isolated execution environments. Different teams’ agents may use different identities and access privileges; a protocol for reaching a tool does not settle who should be allowed to use it.

Gilbert cautioned against treating every new foundation project as an established standard. He explained what he looks for: “I’m tracking not your downloads. I’m tracking the diversity.” For buyers, that means examining independent contributors, working integrations and project maturity alongside a vendor’s product demonstration.

AI costs go beyond model inference

Model choice leaves another economic question: How much work must the system do to supply the right information? Agents can repeatedly query enterprise data, retain context and generate monitoring records. Each activity consumes resources, even when the model call itself is inexpensive.

Bianca Lewis, executive director of the OpenSearch Software Foundation, emphasized the risks of committing to a data platform only to discover that pricing, functionality or operating models demand expensive rearchitecture. At September's OpenSearchCon in San Jose, sessions addressed assembling context across systems and preventing agents from accessing information their users aren't authorized to see. 

AWS’s Tia White, general manager of Amazon OpenSearch Service, highlighted the ecosystem of builders and service providers around OpenSearch, while Intel demonstrated hardware optimizations contributed upstream rather than through a proprietary distribution. For platform and data teams, evaluating open platforms means looking beyond retrieval latency to repeated requests, storage and monitoring costs, governance, and long-term flexibility. 

OpenShift 5 brings AI into cluster operations

Infrastructure costs also include the work required to operate the platform. Red Hat's OpenShift 5 announcement introduces AI-assisted tools for managing Kubernetes clusters. In a pre-event briefing, Chief Product Officer Ashesh Badani described AI-assisted analysis ahead of cluster upgrades, remediation guidance and dashboards created with natural-language commands.

Badani also described MCP integration and support for external orchestration and agent workflows. That would extend the ways customers operate the platform alongside its console and existing automation. For buyers, the relevant question is whether the integration reduces the effort needed to operate their particular environment.

Red Hat says confidential GPU support is generally available on supported bare-metal systems, while its agent sandbox remains in technology preview. The company also outlined a three-year major-release cadence intended to give customers a more predictable platform lifecycle.

When an agent requests a change, the platform needs to establish its identity, constrain what it can touch and provide a way for an operator to intervene.

For an operations pilot, measure upgrade preparation time, manual intervention and policy exceptions before expanding access. A more autonomous workflow should still leave evidence of which actions were taken, which failed and how much it cost to recover.

Stephen Watt, Red Hat's distinguished engineer and vice president of the Office of the CTO, described an internal pilot combining self-hosted open-weight inference, a gateway and developer budgets. The team had to operate the inference service and encourage engineers to use an alternative when it was sufficient for the task. In a published account, Red Hat says the deployment reduced effective token spending by 40% to 70% compared with previous commercial-model pricing, depending on caching and routing.

However, those savings came with new operating work. Buyers should ask who will run the service, how policy will be enforced and how users will adopt it. The answer belongs in the cost calculation alongside hardware and inference charges.

The strategic beat: How to tell whether open infrastructure saves money

The October survey does not establish that open source improves ROI; that is a business case each deployment must demonstrate.

Platform teams should start with one specific AI workflow and work with the person responsible for it to define what success looks like and how much human intervention is acceptable. Have the data team identify authoritative information and access rules. Ask platform engineering to instrument the workflow and measure its operating costs, latency and repeated work, and involve finance in agreeing on the baseline and validating the benefit.

Then change one component at a time. Test another model, serving configuration or retrieval approach while holding the quality requirement constant. Include integration effort, support and human review in the comparison. Lower request prices won't save money if the results require more human review or rework.

At this week's KubeCon + CloudNativeCon North America in Salt Lake City, where the AI Inference + Agentic track covers model serving and AI operations, ask vendors and maintainers to show which interfaces are stable, which improvements go upstream and where commercial support begins. Bring those answers back to the workflow’s economics. The next platform should earn its place by making useful work more dependable or economical.

VentureBeat has been and might be in the future a paid media partner for vendors mentioned. Neither those vendors nor other sponsors have editorial control over content on VentureBeat, VentureBeat Intelligence, or VentureBeat Research. Perspectives and opinions are those of VB Intelligence.