There’s been a lot of talk recently about AI governance; however, much of it focuses on what happens after an AI system has already acted, with organizations monitoring its use and reviewing its output. The problem is that the decision about what the AI tool or agent can access happens earlier, and it usually gets less review than what happens after.
The Decision That Happens Before AI Acts
Before an AI tool or agent generates any code, someone has to decide what it’s allowed to access, including repositories, data, external tools, and what actions it can take without a human in the loop.
This decision often gets made once, early, and isn’t always revisited. Microsoft’s own security team has described this pattern directly, noting that a team will provision an agent with broad access because it is quick, and that access persists even as the agent’s role expands, without anyone going back to narrow it.
After that access is granted, most of a governance program’s attention shifts to what the tool produces, and the access decision itself is seldom revisited.
What Design-Time Governance Involves
With design-time governance, the access decision is defined and checked before a tool or agent goes live.
In practice, this includes mapping out what an AI feature or agent can reach as part of the architecture decision, before any code generation. A useful way to do this is to threat-model an agent’s permissions and system boundaries against its task, rather than granting broad access because narrowing it later feels like it can wait.
It also involves documenting agent access as a governance artifact and keeping it current, in the same way a data flow diagram or an access control list would be for any other system.
It’s best to do this when a feature or agent is designed, and again whenever its task or scope changes, rather than continuously as commit review does. It needs the kind of threat modeling most security teams already do for other parts of the architecture.
Threat Modeling an Agent’s Access
When it comes to agent permissions, teams usually start with what the agent requests, check it against a least-privilege standard, and remove anything that looks excessive. That approach is useful, but it starts from the wrong end by asking whether the requested access is appropriate rather than what could go wrong if it were misused.
Threat modeling starts from the opposite direction, beginning with the ways access could fail or be abused and working backward to what the agent should be allowed to access. For an AI agent, teams should think through what data flows it introduces that did not exist before, what the damage is if its credentials are compromised, and what happens if it receives an instruction from an untrusted source, such as a webpage it reads or a comment on an issue it is resolving.
This is also where prompt injection fits. An agent that reads untrusted content as part of its task can be manipulated into acting on that content alone, without any compromised credentials. The agent’s code can be working as intended, while the surrounding system still leaves an attacker with too much room to act. This risk sits outside what a conventional code review is designed to address.
Teams should already be running structured threat modeling using frameworks such as STRIDE, so applying that same practice to what an AI agent can reach is extending it, not something new.
Four Ways to Scope Agent Access
Threat modeling an agent’s access does not prevent an attacker from attempting a prompt injection. It determines whether that attempt has anything to work with. Scoped correctly, an injected instruction can have no privileged action available to it at all.
These are a few ways to scope an agent’s access:
-
Separate what an agent can read from what it can change. An agent that reads untrusted content to diagnose a problem may not need write access to the same systems.
-
Issue the agent its own scoped credential instead of inheriting a developer’s or a service account’s broader access. A compromise then stays contained to what that credential can reach.
-
Restrict which tools and functions the agent can invoke; for example, it doesn’t need every operation that an API supports.
-
An agent can prepare a change, but actions such as deleting data or modifying CI/CD configuration should still require someone to approve them.
A Worked Example
An agent connects to a repository to identify failing tests and propose fixes. A permissions checklist might conclude that it needs read and write access to the repository and approve that setup.
A threat modeling review asks a different set of questions. Could test output contain an instruction designed to influence the agent? Does the agent need write access to the entire repository to modify the failing test? Could it change CI/CD configuration? What else could it reach if its credentials were compromised?
That gives the agent only the access it needs for the job. It can write to the test files, but can’t change CI/CD configuration unless the task requires it. It also uses its own credentials rather than a broader service account, so a compromise doesn’t automatically open up everything that account could access.
It’s the same agent, performing the same task, but the difference between the two versions is only apparent if someone asks what issues could happen before granting the access, rather than after.
How ThreatCanvas Helps Threat Model AI Access
AI governance needs to account for the decisions that govern what an AI system can access and do, as well as what it produces. For AI agents in particular, defining those boundaries before deployment gives security teams an opportunity to identify excessive access and potential attack paths while they can still be addressed.
ThreatCanvas brings structured threat modeling into the design stage, before an AI feature or agent is deployed. It automatically generates threat models from diagrams, text descriptions, infrastructure-as-code templates, or a code repository. Risk templates aligned to the OWASP Top 10 for LLM Applications and the OWASP Top 10 for Agentic AI turn those established risk categories into something a team can apply directly, rather than starting the threat model from a blank page.
This works alongside the rest of SecureFlag’s platform, including hands-on training for reviewing AI-generated code and hands-on labs covering application security, AI security, cloud security, and secure coding across 70 programming languages and frameworks. It supports AI governance before and after an agent is deployed.
See how hands-on training and ThreatCanvas can help teams manage AI risks from design through development.
