Most conversations about AI security still orbit the model itself: Is the large language model (LLM) aligned, can it be jailbroken, does it hallucinate under pressure? Those are real questions. But as organizations move AI agents and LLM-powered workflows from pilot projects into production, a different category of risk is emerging, one that has almost nothing to do with the model's weights and everything to do with the system built around it.
The model is rarely the weak point. The workflow is.
The shift that changes the threat model
For the past two years, most enterprise AI security discussion has assumed a fairly narrow interaction pattern: A user types a prompt, a model returns text, a human reads it. That threat model is already obsolete. Production AI systems today read from live data sources, call external tools and APIs, write to databases, trigger downstream automations, and, in a growing number of cases, take actions with no human in the loop at all.
Each of those capabilities is also an attack surface that didn't exist in the "chatbot" era.
Prompt injection through connected data
When an agent retrieves content from a CRM record, a support ticket, a scraped webpage, or a shared document, that content becomes part of its context, and an attacker doesn't need access to your model to manipulate its behavior; they only need access to something the model will eventually read. Instructions buried in a PDF, an email signature, or a product review can hijack an agent's next action just as effectively as a crafted prompt typed directly into a chat window.
Over-permissioned tool calls
Agent frameworks increasingly grant models the ability to call functions: Send an email, query a database, modify a record, execute code. Many of these integrations are built with development-speed permissions, broad API keys, service accounts with far more access than the specific task requires because it's the fastest way to get a demo working. In production, that same permission model means a single manipulated prompt or a single reasoning error can cascade into a real-world action: A wrong record deleted, a message sent to the wrong recipient, a payment triggered.
Fragile trust boundaries between tools
Multi-tool and multi-agent pipelines pass outputs from one component into another, often without re-validating what's crossing the boundary. A model summarizing an untrusted document and handing that summary to a second agent that has write access to a production system is, functionally, letting an unauthenticated third party influence that system even though no human ever explicitly granted it that trust.
Missing observability
Traditional application security assumes you can log a request, trace a call stack, and reconstruct what happened after an incident. Agentic workflows often can't offer that. Reasoning traces are non-deterministic, tool-call sequences vary from run to run, and many teams don't yet log intermediate agent decisions at a granularity that would let them answer a basic incident-response question: What did the system actually do, and why?
Automated decision flows with no circuit breaker.
As autonomy increases, an agent doesn't just draft a response; it sends it. The agent doesn't just flag an anomaly; it acts on it. The cost of a single bad decision compounds, especially when there's no rate limit, approval gate, or kill switch between "the model decided" and "the action happened."
None of these are model problems. You can swap in a more capable, better-aligned LLM and they all persist, because they live in the plumbing, not the weights.
Why this is a governance problem, not just an engineering one
Part of why workflow-level risk gets underinvested is organizational: AI security questions tend to land with machine learning (ML) or data science teams, who are equipped to evaluate model behavior but are not necessarily positioned or funded to own integration security, access control, or production observability. Meanwhile, security and platform engineering teams, who have decades of institutional experience securing exactly this kind of system (services calling services, credentials with scoped permissions, audit logging, incident response), often aren't looped in until an agentic workflow is already live.
The organizations scaling AI successfully tend to close that gap early. They treat an AI agent the way they'd treat any new production service with write access to real systems, which in practice means a fairly unglamorous checklist:
Least-privilege access by default. Tool and API permissions scoped to exactly what a given task requires, not to what's convenient during development. If an agent only needs to read a customer record, it shouldn't hold a credential that can also write to it.
Sandboxing before autonomy. New tool integrations and agent capabilities get tested in an environment where a bad decision is cheap, with a deliberate, monitored graduation path to production access not shipped directly with full permissions because the demo worked.
Human approval gates on high-consequence actions. Autonomy is not binary. A workflow can let a model draft an action while still requiring explicit approval before anything external happens: A refund, an email to a customer, a change to a live system, with the approval threshold set by the blast radius of the action, not by how confident the model sounds.
Logging at the decision level, not just the output level. Capturing what an agent read, what tools it called, what arguments it passed, and why, so that after an incident, the question "what happened and why" has an answer that doesn't require guessing.
Treating third-party data as untrusted input. Any content an agent ingests from outside your own verified systems — a webpage, an uploaded file, an email — should be handled with the same suspicion as user input in a traditional web application, because, functionally, that's exactly what it is.
Explicit circuit breakers. Rate limits, anomaly detection, and manual override capability on automated actions, so a single bad reasoning chain can't compound into thousands of bad actions before a human notices.
The conversation that needs to change
None of this is exotic. It's the same engineering discipline that's governed production software for two decades: Scoped permissions, input validation, observability, staged rollouts, human review of consequential actions. What's new is that a probabilistic, occasionally unpredictable model is now sitting inside that system, making some of the decisions about what happens next.
That's precisely why the workflow around the model deserves more scrutiny than it's getting. A more capable model doesn't make an over-permissioned integration safer. A better-aligned LLM doesn't fix a missing approval gate. As AI agents take on more autonomous, consequential work inside real businesses, the organizations that get burned won't necessarily be the ones using a weaker model; they'll be the ones that never asked whether the system around their model was built to be secured at all.
Adithyan RK is co-founder and CEO of Hyring
Welcome to the VentureBeat community!
Our guest posting program is where technical experts share insights and provide neutral, non-vested deep dives on AI, data infrastructure, cybersecurity and other cutting-edge technologies shaping the future of enterprise.
Read more from our guest post program — and check out our guidelines if you’re interested in contributing an article of your own!