The interview transcript below has been edited for length and clarity.
Rob Strechay: If an AI agent goes wrong, can you stop it and keep the business running? I'm Rob Strechay with VentureBeat Intelligence. In our August pulse survey, half of the respondents said their organizations had agents in productions. Among that group, that half, 61% reported at least some shared credentials amongst those agents.
Our September research adds another concern. Only 14% of respondents answering the revocation question said a named owner could revoke an agent's access within minutes. That raises a practical question: what needs to be secured before a model or agent ever runs, and how do those infrastructure protections work alongside guardrails and application controls?
Joining me is Nir Sharoni, who's the head of security products at Nebius. We'll explore how enterprises can build security, resilience, and control into the infrastructure supporting AI, and what GPU confidential computing brings to that conversation. Welcome in, Nir.
Nir Chervoni: Thank you for having me, Rob. Happy to be here.
Rob Strechay: Yeah I think this is one of those really exciting questions. It's been very topical now for a little bit because people are really, focusing in on security. But Nir, from your perspective, again, leading security products at Nebius, what does an enterprise need to secure before a model or agent ever runs, and how should infrastructure protections really work alongside those m- model guardrails and application controls?
Nir Chervoni: Yeah. When agents share credentials, you lose two things at once. One, you cannot tell which actor did what, and you cannot limit the damage when one is compromised. So before anything runs, an enterprise needs few foundations. Every agent s- gets its own identity the same way a human user does.
Credentials eventually are short-lived and scoped to the task, and they're kept in secrets manager rather than in prompts, config files, or environmental variables. Encryption keys stay under the, the customer's control, and there's a complete audit trail that feeds into SIEM. On the layers, infrastructure controls are the floor.
They hold no matter how the model behaves, and that's an important one. Now, guardrails minimize risk, but they offer statistical protection rather than total guarantees. App- application controls carry the business logic. My rule is pretty simple: never rely on a guardrail to enforce something an access policy could enforce deterministically.
That's why at Nebius we've invested first in identity and access management, key management services, secrets management, audit logs, et cetera. And I'd say that the, the- these are the boring layer or components that makes everything above trustworthy.
Rob Strechay: Yeah, like I said, I think, starting at the infrastructure and building it in is a must.
I agree with that totally. I think another place, and I think you're talking about this at GTC Berlin in a panel, is really confidential computing coming to GPUs. We've had it in CPUs for a little bit now, but what changes when you apply it to GPU based AI workloads, and what protection does it provide to enterprise customers while their data and models are in use?
Where does that protection stop, and what do you expect the panel to clarify for customers evaluating confidential computing at the GPU layer?
Nir Chervoni: As you said, confidential computing used to stop at the CPU. Nowadays, the GPU joins the trust boundary. Data moving between CPU and GPU is encrypted, and the GPU itself can be attested.
Now, in practice, a customer's data and model weights stay protected while they're being processed in some implementations, including from the cloud operator, the hypervisor, and our own administrators. Attestation is one of the most important components, and I'm going to elaborate about that in the panel itself as well.
Because it lets the customer verify the environment before releasing their keys, so trust is something they check rather than something they assume. Now, it has limits, though. Confidential computing doesn't fix vulnerabilities in your application. It doesn't stop prompt injection or govern what the model outputs.
It doesn't help if a legitimate identity is compromised. It also only as strong as whoever verifies the attestation itself. On the panel I want to make the point that this is tool for drawing a trust boundary, not a blanket over everything. Customers should ask themselves who verifies the attestation, what the performance cost is, and how mature multi-GPU support is at scale.
Rob Strechay: Yeah I agree. I think that when you start to look at how complex it is, you gotta start with a good foundation. I think confidential computing coming to GPUs makes a lot of sense in that way because it really helps to I guess you could say limit the blast radius of things and start to compartmentalize how you get access to certain data and really keep it locked up a little bit more than, people have been doing.
But again, building off of what was done in CPU. So it's really, I think a, a great thing and a great panel to be seen, in the next week here. Recently though, independent research into GPU cloud providers found cross-tenant risks rooted in basics: patch management network exposure, shared control planes.
What should a CISO ask any AI infrastructure provider to verify tenant isolation
Nir Chervoni: Well, the research you've mentioned is right, that the risks come from the basics. So actually simply ask about the basics. Things like what happens to bare metal GPU node between tenants? Is the firmware re-flashed?
Is GPU memory scrubbed? Is the integrity check verified? That- that's one category of questions. Then is the management network, meaning like things like BMCs or out-of-band access, physically or logically unreachable from tenant workloads? Or things like which parts of the control plane are shared?
How its authorization enforced per tenant? How far could a compromise of the controls plane spread? Has tenant isolation specifically been pen tested by a third party? How's the high-speed fabric, whether InfiniBand or RoCE, partitioned between tenants? What I'm trying to say is that if the provider can't answer these questions with evidence, that tells you what you need to know.
Rob Strechay: Yeah. I like how you put it that this is the basics, right? The knitting of really refreshing the environment and making sure it's up to date. You talked about it s- software supply chain attacks that are going on and stuff like that, but, that's out of that control, but making sure that you have things in place to make sure you're putting the right software and patch levels and things on there at the basics.
I think, as we discussed, previously one of the things that I'm really seeing a, an uptick in is really sovereign AI. And sovereign AI involves more than where data resides. How should customers think about who can access their data, how it is processed, and who controls the protections around it?
And can you illustrate how those requirements shape a secure deployment from an AI perspective?
Nir Chervoni: Yeah. First of all, let's distinguish between residency and sovereignty. Residency answers only one question, where the data sits. Sovereignty answers three more questions. Who can access it? Like the, the, that covers provider staff, sub-processors, which legal jurisdictions can compel access.
The other question would be how it's being processed, like things like zero retention, no training on customer data, et cetera. And the, the last one would be who controls the protections? And that's the one most people miss actually. Now, take a regulated European organization as an example. Their workload run in an EU region on isolated, dedicated capacity.
Their keys sit in a KMS they control, and ideally the keys are released only after the environment passes attestation. Any access by provider staff is just-in-time, approved, and logged directly into the customer's own SIM. Inference is zero retention, and the jurisdiction rule covers backups and support access too, not just primary storage.
The test is where the customer can verify every one of those points themselves without having to take our words for it
Rob Strechay: Do you, as, as we zoom back out, you talk to a lot of organizations. Do you see organizations more ... that are outside those regulated industries asking more questions about sovereign AI because they wanna operate in that manner as well?
Nir Chervoni: In most cases, I'm g- we're getting these questions from more highly regulated companies, like prospects and customers, and less from those who has, let's say, least obligations to the regulator. Because otherwise, for them, they mostly like to tick the mark with show me your SOC 2 reports, show me your ISO 2701, et cetera, et cetera.
Rob Strechay: What is, kind of final thought here, what is your number one takeaway that you want CISOs that are out there to think about as they approach? Because we see great interest in AI specialized clouds like Nebius, and what should they be thinking about as they try to figure out where to place their AI workloads from a security perspective?
Nir Chervoni: I think the basics are still critical when you're considering an AI cloud, even though it's a bit different than the traditional CPU cloud. Because identity and access managent and access management are still valid and relevant, just need to be a bit more enhanced or improved or AI particular.
And things like how do you store your secrets, things like how do you audit and provide proper loggings for any actions that have been taken, whether authorized action or unauthorized action, things around how do you protect your data, whether the protection is effective, whether the protection is something that you can verify or you can allow your customers to verify by themselves instead of promising as a service provider that you give it to them.
So these are the things that I would expect CISOs to check while evaluating an AI cloud, and not very different than when they're evaluating a traditional cloud.
Rob Strechay: I couldn't agree more. I think, again, it's sta- starting with the basics and covering those and making sure that you're aligned makes a lot of sense.
So Nir, hey, thanks for coming on board. I really appreciate this. I think, again, really looking forward to catching up after GTC as well, and we'll have a lot of fun unpacking that.
Nir Chervoni: Sure. I'm more than sure it's gonna be great there on GTC, and thanks for having me here today.
__