Infrastructure

Enterprise AI demand is real, but it’s landing on frontier models, not the GPUs companies already hold. The next savings are in tokens, context and agents.

Companies risk spending on more AI infrastructure without knowing whether it will solve their existing problems or deliver a return.

AWS’s published announcement confirms support for gpt-oss-20b, gemma-4-31B-it, Qwen3.6-27B and Qwen3.8-27B.

Under the Apache 2.0 license, other companies can incorporate the Coral NPU architecture into commercial processors without paying Google licensing fees or royalties.

GPUs running large language model inference can spend up to 60% of their compute recalculating context they already processed, says Alan McSeveney, field CTO at WEKA. Speaking with VentureBeat lead analyst Rob Strechay at the latest stop in the VentureBeat AI Impact Series, McSeveney said the waste goes back to how large language models hold conversation history in GPU memory, arguing that fixing it lets companies serve several times more users on hardware they already own. The efficiencies WEKA customers struggle with most show up in infrastructure, separate from the application metrics each business tunes for its own domain.

Presented by Virtana

Presented by EDB

Nvidia's open-source OpenShell enforces agent permissions on existing hardware, while Sentry adds protection for higher-risk workloads.