Production AI agents should be secured as delegated actors, not treated as chatbots. The minimum safe design gives every agent a bounded identity, least-privilege tools, untrusted-input handling, explicit approval gates, tamper-evident logs, and a tested way to stop it. The central question is no longer only what the model can say; it is what the whole agent system can do.
What changes when a model becomes an agent?
A conventional model returns content. An agent can plan, retrieve data, call software, modify records, send messages, or trigger another agent. That creates a chain of authority: user to agent, agent to model, model to tool, and tool to enterprise system. A control failure at any link can turn an untrusted instruction into a business action.
The OWASP Top 10 for Agentic Applications 2026 frames the risk around agent goal hijack, tool misuse, identity and privilege abuse, supply-chain vulnerabilities, unexpected code execution, memory poisoning, insecure inter-agent communication, cascading failures, trust exploitation, and rogue agents. OWASP’s AI Agent Security Cheat Sheet translates those risks into engineering controls.
What should a production threat model cover?
Model five boundaries before deployment: instructions entering the context; data entering retrieval or memory; tools exposed to the agent; identities used to call those tools; and outputs that affect people, money, data, or infrastructure. For each boundary, record the trusted source, possible attacker, maximum consequence, preventive control, detection signal, and recovery action.
1. Give the agent a bounded identity
Do not let an agent inherit a developer’s broad credentials. Give it a workload identity with the smallest permissions required for the current task. Prefer short-lived credentials, scoped tokens, row-level access, and separate read from write permissions. A support agent that reads an order should not automatically be able to refund it.
2. Treat retrieved content as data, not authority
Documents, emails, webpages, tickets, and tool outputs can contain hostile instructions. Delimit retrieved content, preserve provenance, filter active content, and prevent it from changing system policy. Prompt injection cannot be solved by a single instruction; containment depends on the permissions and validation surrounding the model.
3. Put policy between the model and every consequential tool
A model’s proposed action is a request, not authorisation. A deterministic policy layer should validate the caller, action, resource, parameters, rate, and current risk state. Payments, deletions, credential changes, regulated decisions, and external communications need additional approval or a separate execution service.
4. Constrain memory and agent-to-agent communication
Separate working from durable memory. Validate what is written, attach source and time metadata, and make sensitive memory expire. Authenticate messages between agents and define which agent may delegate which task. Otherwise, poisoned memory or a compromised specialist can silently influence future runs.
5. Preserve an investigation-grade trace
Capture the request, policy version, model and tool versions, retrieved sources, proposed action, approval, tool result, and outcome. Protect sensitive fields, but keep enough lineage to answer: who or what acted, under whose authority, on which data, and why?
Where should humans approve an agent’s work?
Human approval belongs at irreversible or high-consequence boundaries, not after every model turn. Low-risk, reversible actions can run automatically within tight limits. Medium-risk actions can require sampled review or exception approval. High-risk actions should require an authorised person with the context needed to decide.
What must be tested before launch?
- Direct and indirect prompt injection across every retrieval source.
- Tool argument manipulation, excessive requests, and unauthorised resources.
- Credential expiry, revocation, and privilege-boundary failures.
- Poisoned memory, malformed agent messages, and compromised dependencies.
- Partial outages, retry storms, duplicated actions, and cascading failures.
- Kill switch, rollback, evidence preservation, and incident ownership.
OWASP’s Q1 2026 exploit round-up reinforces why teams should test observed attack patterns, not assess only architecture diagrams.
What is the production gate?
Before autonomy increases, require evidence that the agent stays within task scope, cannot exceed its permissions, contains hostile context, produces reviewable traces, fails safely, and can be stopped quickly. Cognis uses this boundary-first approach in AI agent and automation engineering: autonomy is earned by measured reliability, not granted because a demo works.