What to establish before connecting an AI agent to a sensitive system
The facts, boundaries and tests to have in place before an AI agent is allowed to act on a system that matters.
Connecting an AI agent to a system is a decision to let software act on the organization’s behalf, through interfaces that were usually designed for people or for other, more predictable programs. The agent will use whatever the interface permits, in combinations nobody wrote down, at a pace nobody planned for. The questions to answer first are not about the model. They are about the system.
This article sets out what should be established, in what order, and what each step does and does not give you.
First, a distinction
Three things get mixed up in these conversations:
- Capability. What the system can be made to do through the interfaces the agent will have. A fact about the system, discoverable by analysis.
- Authorization. What the organization has decided the agent may do. A decision, made by the people accountable for the system. Analysis can inform it. Analysis cannot make it.
- Behavior. What the agent actually does when running. Partly a property of the model, partly of the tools it is given, partly of the inputs it meets. Testable within limits, never fully guaranteed.
Establishing capability is necessary. It is not sufficient. A complete map of what a system exposes tells you nothing about whether the agent should be allowed to use any of it, and a policy decision does not by itself make the agent comply.
Establish what the system exposes
Start with the interface the agent will actually use, not the one the documentation describes.
- Enumerate every operation reachable through that interface. For an HTTP API, every route the agent’s credentials can call, including the ones not in the spec. For a browser or desktop interface, every action available in the states the agent can reach.
- For each operation, record the effect: read, create, update, delete, send, pay, grant. Label each as observed or inferred, and verify the consequential ones with tests.
- Record the blast radius. Which other systems does an operation touch? A “cancel order” call that also triggers a refund and a customer email has three effects, not one.
- Note the operations that are reversible and the ones that are not.
The output is a capability map: operations, effects, dependencies, evidence. Open items are expected. “We could not determine whether PATCH /accounts can change the billing owner” is a finding that shapes the authorization decision.
Establish the boundary
With the map in hand, the owning organization decides what the agent may do. Write it down as a list of permitted operations, with conditions, rather than as a list of forbidden ones. Anything not on the list is out.
For each permitted operation, decide:
- Under what conditions. Amount limits, customer segments, time windows, environments.
- Whether it requires a human approval step, and who gives it.
- What happens on failure: retry, stop, escalate.
- How the action is attributed, so the audit trail distinguishes the agent from a person using the same credentials.
Then enforce the boundary somewhere the agent cannot reach. Scoped credentials, a gateway that only exposes the permitted operations, server-side limits. Instructions in a prompt are a request, not a control.
Establish that the tools match the system
The agent experiences the system through its tools: an SDK, a set of functions, a browser. If the tool’s description of an operation differs from the operation’s real behavior, the agent is acting on a false belief. This is where conformance tests earn their place.
Before connecting:
- Every tool the agent has maps to a verified operation on the capability map.
- The tool’s parameters, required fields and error handling match the system’s actual behavior, confirmed by tests against the real system.
- Error responses are surfaced to the agent in a form that distinguishes “this failed, do not retry” from “this failed, retrying is safe”.
- Idempotency is handled by the tool, not left to the model. An agent that retries a payment because the first response timed out should be unable to pay twice.
Establish how you will know
Observability for agents is observability for the system, plus attribution.
- Every action the agent takes is logged with the agent identity, the task it was working on, the inputs it saw, and the operation it called.
- Rate and volume are watched. Agents fail fast and loud, or slowly and quietly; both need a signal.
- The capability map and conformance tests are re-run when the system changes. An upstream deployment that adds an operation, or changes the semantics of one, changes what the agent can do without anyone changing the agent.
Test the agent against the boundary
With the system mapped, the boundary enforced and the tools verified, the agent itself can be tested. Give it tasks that invite it to exceed its permissions, mishandle an error, or act on an ambiguous instruction, and confirm that the enforcement layer, not the model’s good intentions, is what stops it. Record what it attempted. Attempts that were blocked are findings too: they tell you what the agent would do with wider permissions.
These tests reduce risk. They do not eliminate it. An agent that behaved well on a thousand tasks has not been proven to behave well on the next one. The boundary is what holds when the behavior does not.
A checklist
Before the connection goes live:
- Capability map complete for the interface the agent will use, with open items listed.
- Authorization decision written, owned, and expressed as permitted operations with conditions.
- Enforcement in place outside the agent’s reach.
- Tools conformance-tested against the real system.
- Idempotency and retry handled below the model.
- Attribution, logging and rate signals in place.
- A plan to re-verify when the system changes.
- Agreement on who can revoke access, and how fast.
Where Foundry fits
Foundry produces the capability map and the tested interfaces: what a system lets an agent do, verified operation by operation, with SDKs and conformance tests that keep the agent’s tools faithful to the system, and change reports when the system moves. It informs the authorization decision; it does not make it. See the page for AI agent builders.