Presented by VAST Data

Sovereignty rules, regulation, security review, cost predictability, and air-gapped operations keep the most valuable enterprise data out of the public cloud. In order to bring the benefits of AI to these highly valuable datasets, frontier and open-source models now have to run where enterprise data already sits. However, concerns over securing their model weights in datacenters they do not control have so far kept model providers from delivering this capability.

“Confidential AI” addresses both sides of that equation: enterprises retain control of their data, while model providers keep their proprietary weights protected when models run in infrastructure they don’t own. That protection has to cover data at rest in storage, in transit across the network, and in use in memory, with the environment verified before encryption keys are released.

"Organizations like banks, government agencies, and health care providers have a lot of data that was never intended to move to the cloud," says Phil Manez, VP of strategic initiatives at VAST Data. "We're at the point now where the opportunity cost of not having these advanced AI models access that data is coming to a head."

VAST recently introduced DataEnclave, a confidential AI capability within the VAST AI Operating System designed to enable enterprises to run advanced AI where their data resides while protecting model providers’ proprietary weights. That capability becomes critical as enterprises look to bring AI to data that has never left their own environments.

Until now, much of enterprise AI has run on whatever data happened to be portable, never touching a proprietary record. Hitting the next level of value depends on tapping into those records that never left the building. Reaching it means running frontier and open-source models inside environments the enterprise controls.

A model in an enclave sees only what the enterprise sends it

A model can run on-premises as soon as someone deploys a container against a GPU. But the model inside the enclave never goes looking for proprietary records itself; the data is sent to it. A retrieval application outside the enclave searches the enterprise’s documents, assembles the relevant excerpts into a prompt and sends that prompt to the model’s inference API, which makes the prompt the only enterprise data the model ever sees. The provider that owns the model gets no administrative path into the enclave to read that prompt or the response that comes back.

"I need to have ways to control what data the AI can access, and get visibility into what the AI was doing," Manez says. "If you can't do either of those, just allowing the model to run on prem doesn't give you much."

Why model builders won't put their weights in a customer's data center

For model builders, the weights in an LLM represent billions of dollars of training investment as well as valuable intellectual property built on top of it, so shipping them into a customer’s data center puts the company’s core asset inside somebody else’s building. Anyone who can read GPU memory while a model runs could copy the weights and stand up the model somewhere else while its builder sees no return.

The worry goes beyond revenue. Builders also have to be able to stop misuse, as Anthropic showed when it disclosed in September that it had disrupted attempts to use Claude in work that could support biological-weapons development, an enforcement action open to a provider only while it still controls where its model runs.

Confidential computing and attestation protect a model in use

It's been possible to encrypt data at rest and protect it in flight for a long time. The biggest challenge has been securing it in memory while a machine works on it.

Confidential computing addressed that problem for CPUs by isolating working memory from the host operating system, hypervisor, and administrators with root access. But AI workloads also use GPUs, where training and inference put weights, prompts, and intermediate results into memory while they’re in use.

"You have to have a way to validate that the model is running in a secure environment before it's allowed to run — that's the attestation part," Manez says. "Only when you've proven you're in that secure environment do you get the keys to unlock that proprietary information. Then you have the audit trail that proves when this model was decrypted and how it was used."

Model builders want the mirror image of an enterprise audit record, showing that their weights were decrypted only in environments they cleared. That control outlives the deployment because the attestation server belongs to the model’s owner, and every launch, including each new replica, has to pass it again.

VAST has been solving the enterprise half for years but, until now, could offer it only for open-source models or models that already shipped with an on-premises option, which ruled out the strongest cloud-only models.

Extending one operating model to sovereign, on-premises and air-gapped AI

VAST DataEnclave applies that architecture by running the model inside a hardware-isolated virtual machine. The model’s owner releases the decryption keys from its own attestation server only after hardware-signed evidence shows that the machine matches the policy the owner set. Neither VAST nor the company operating the hardware ever holds another party’s keys.

DataEnclave builds confidential virtual machines that span the CPUs and GPUs and protect the transfers between them, while on newer GPU platforms encrypted NVLink covers traffic between GPUs.

DataEnclave adds two pieces to the VAST DataEngine, which handles event-driven computing, secure model deployment and accelerated inference. The secure runtime launches each workload in its own confidential virtual machine on the infrastructure where the data sits. For sites with no outside connection, the attestation and key-management systems can run on premises alongside the workloads they verify. The operating model stays the same whether the workload runs in a public cloud, sovereign cloud, customer data center or air-gapped facility.

"Translating the different controls, access requirements, and what audit looks like across all of those is very difficult, and it leaves opportunities for mistakes," Manez says. "With a common operating model, I create policies one time, and then I move those policies through those environments."

Permissions also have to survive the trip from a file into a vector index. VAST enforces one set of access controls on a file, an object, or an image, and on the vector that represents it, which stops a retrieval pipeline from handing someone the contents of a document they cannot open.

Those same controls decide what reaches a model inside an enclave. Search and prompt assembly happen on the enterprise’s own systems, so the model receives a prompt built only from records the requester can already open, and the enterprise’s existing egress rules govern where the enclave can send anything. The runtime logs which application and version launched, on which node and under which configuration, into the VAST DataBase, while the attestation service keeps its own account of which enclaves it verified and which keys it released, so neither party has to take the other’s word for what happened.

Model portability sets the size of a provider's addressable market

A model that runs only in its provider’s cloud can sell only to buyers that are free to put their data there, which limits its addressable market before anyone evaluates the model itself.

"If you can only run in the public cloud, your sales roster is heavy on consumer and non-regulated industries," Manez says. "I'm missing out on government. I could potentially be missing out on all of Europe. I'm missing out on health care and financial services customers, which can be a lucrative market for these model builders if they meet the privacy and regulation requirements."

VAST describes the longer-term version as an ecosystem where model builders, AI clouds, and infrastructure partners publish into one trusted foundation, so a customer's location stops determining which models it can license.

"The operating and revenue model is becoming the biggest inhibitor," Manez says. "The model IP has evolved very quickly and it's moving all the time. But without the operating model that satisfies the requirements of the enterprise, AI will feel like a hype bubble. There's way more to it than a VM technology or a data encryption technology or a network technology. It's the entire infrastructure and ecosystem around the model."

Sponsored articles are content produced by a company that is either paying for the post or has a business relationship with VentureBeat, and they’re always clearly marked. For more information, contact sales@venturebeat.com.