How to close the evidence gap when your auditor asks about AI agent controls
SOC 2 was designed for deterministic software. LLM-powered agents create evidence gaps across five Trust Services Criteria that traditional controls don't address.
How to close the evidence gap when your auditor asks how you control an AI agent's behavior
SOC 2 Type II audits evaluate whether your controls operated effectively over a defined audit period — typically 6 or 12 months. For traditional software, this means sampling access logs, change records, and monitoring alerts across the period and verifying that controls were consistent and functioning.
LLM-powered applications break this model. An AI agent's behavior is not deterministic — it depends on inputs, context, and model state that vary at runtime. The tools it can invoke aren't a static access control list. Its "change events" include model updates that don't go through a traditional change management process. And "processing integrity" for a system that can hallucinate has no direct precedent in standard Trust Services Criteria.
This post covers the five Trust Services Criteria most affected by LLM deployments, the specific evidence gaps that auditors are increasingly flagging, and how to produce continuous control evidence that satisfies Type II requirements.
Traditional approach: RBAC configuration screenshots, access review records, user provisioning logs.
AI agent gap: The agent's access is not defined by a role in an RBAC system. It's defined by which tools the agent can call, which external systems those tools connect to, and what parameters the agent chooses at runtime. A traditional access review cannot capture this.
What auditors now ask: "How do you know the AI agent only accessed the data it was authorized to access? Can you show me a log of every data access decision it made over the audit period?"
Traditional approach: SIEM alerts, anomaly detection on access logs, performance monitoring dashboards.
AI agent gap: Standard SIEM rules are designed for deterministic access patterns. An AI agent that accesses a normally-accessed database via an unusual query path may not trigger any anomaly alert — because the access itself is authorized, even if the context suggests it shouldn't be.
What auditors now ask: "How do you detect when an agent is behaving outside its intended scope? What controls alert you to unusual agent behavior?"
Traditional approach: Change tickets, approval records, deployment logs, rollback procedures.
AI agent gap: Model providers update base models on their own schedules. A model update changes agent behavior without a change ticket in your system. An agent that passed behavioral testing on model version N may behave differently on version N+1 — and the change never went through your change management process.
What auditors now ask: "How do you manage changes to the AI model itself? What testing validates agent behavior after a model update?"
Traditional approach: Input/output validation, data checksums, error rate monitoring.
AI agent gap: LLMs can produce confidently-stated incorrect outputs — hallucinations — that are structurally valid but factually wrong. Standard data validation catches malformed outputs; it cannot catch plausible-looking errors. Processing integrity for AI requires confidence scoring and behavioral consistency evidence.
What auditors now ask: "How do you ensure the AI agent's outputs are accurate? What evidence do you have that processing integrity controls are working?"
SOC 2 Type II requires evidence that controls operated over the full audit period. For AI agents, this means producing records that are continuous, structured, and attributable — not point-in-time certifications.
| Criterion | Required Evidence | AgentTrust OS Source | Frequency |
|---|---|---|---|
| CC6.1 | Per-session tool access log with permission contract reference | Trust Runtime + Trust Audit | Continuous (every action) |
| CC7.2 | Behavioral anomaly records: escalated and blocked actions | Trust Runtime + Trust Audit | Continuous (every escalation/block) |
| CC8.1 | Re-certification record after each model update or policy change | Trust Certify | Event-triggered (on change) |
| CC8.1 | Change record linking deployment event to certification evidence | Trust Certify certification report | Per deployment |
| PI1 | Confidence score distribution over audit period; human review rates | Trust Audit analytics | Aggregate + per-action |
AgentTrust OS enterprise plans include pre-formatted SOC 2 audit pack exports: a structured JSON + PDF report covering each Trust Services Criterion, with control references mapped to the relevant audit records. Your auditor receives a complete evidence package — not raw log files that require manual analysis.
Trust Certify, Trust Runtime, and Trust Audit generate continuous SOC 2 evidence across your full audit period. Enterprise plans include pre-formatted audit pack exports.
View Compliance Resources →