What Happened
A financial services firm deployed over 40 AI agents across three cloud accounts. Twelve managed customer-facing tasks like account summaries, support ticket triage, and compliance reports. Others handled internal workflows such as code reviews, document extraction, and fraud scoring. A third group operated in a sandbox without formal oversight.
The firm's Cloud Security Posture Management (CSPM) tool flagged 14 IAM misconfigurations but couldn't differentiate between a low-risk development notebook and a production agent with access to personally identifiable information. It recognized compute and storage resources but not agent identity, tool-call permissions, or whether an agent could create sub-agents with elevated access.
Shadow AI worsened the visibility issue. Teams launched agents using personal API keys, routing prompts through external services without telemetry. The security team lacked an inventory of running agents, the data they accessed, or their capabilities.
This wasn't a breach but a posture failure: the infrastructure existed in a state where the organization couldn't assess its own risk surface.
Timeline
The scenario is representative, not tied to a specific incident. The situation is ongoing: organizations deploy AI agents faster than they can manage them.
Which Controls Failed or Were Missing
AI Discovery and Inventory: The firm lacked an AI bill of materials (AI-BOM). It couldn't answer basic questions: How many agents are in production? Who owns them? What data do they access? An AI-BOM catalogs running agents, ownership, data access patterns, and lifecycle stages. Without it, posture assessment is impossible.
Permission and Exposure Assessment: The security team couldn't audit what each AI workload could access or expose, including direct data access, tool-call permissions, API credentials, and external service interactions. The CSPM tool detected infrastructure misconfigurations but couldn't map effective permissions at the AI layer.
Behavioral Baseline and Drift Detection: No system tracked agent behavior over time. The organization couldn't detect deviations from expected patterns, such as tool-call distributions or response latency profiles. Infrastructure drift detection doesn't equate to agent behavior monitoring.
Runtime Monitoring and Enforceable Policy: The firm lacked both pre-LLM and post-execution guardrails. Pre-LLM guardrails should intercept inputs before reaching the model, redacting PII and credentials and blocking prompt injection attempts. Post-execution guardrails should inspect outputs before returning to users or downstream systems. Neither was in place.
Agent Identity Management: The posture system treated all AI workloads as undifferentiated model endpoints. It couldn't distinguish Agent A from Agent B when both called the same tool. Every agent, including dynamically spawned sub-agents, must carry a verifiable identity through every execution step.
What the Standards Require
The EU AI Act requires providers of high-risk AI systems to maintain Technical Documentation (Annex IV) detailing system components, data governance measures, and risk management procedures. Organizations can't produce this documentation if they don't know what agents are deployed.
NIST AI RMF Govern function (GV-1.1) requires policies and procedures to map, measure, and manage AI risks throughout the AI lifecycle. The Map function (MP-2.3) calls for tracking AI system dependencies, including third-party components. The firm's inability to inventory its agents or assess their supply chain represents a direct failure of these functions.
ISO/IEC 42001 Annex A control 6.2.2 requires organizations to maintain an inventory of AI systems, including their purpose, data sources, and operational status. Control 6.2.6 addresses AI system monitoring, requiring organizations to define and implement monitoring activities appropriate to the AI system's risk level.
SR 11-7, focused on model risk management in financial services, establishes the principle that effective challenge requires independent validation. You can't validate what you can't see. The firm's posture gap meant that model risk management couldn't extend to the agent layer.
Lessons and Action Items for Your Team
Build an AI-BOM first: Before assessing posture, you need an inventory. Deploy tools that discover and catalog every model, agent, pipeline, and dataset. The AI-BOM must capture ownership, data access patterns, tool permissions, and lifecycle stage. Update it continuously, not quarterly.
Instrument agent identity at deployment: Assign every agent a distinct, auditable identity before it runs in production. Ensure that dynamically spawned sub-agents inherit scoped-down credentials, not their parent's full permissions. Your monitoring system should trace every tool call back to a specific agent identity.
Define posture policies declaratively: Write agent posture policies specifying allowed tools, denied tools, maximum tool calls per run, sub-agent limits, and human-approval requirements. Enforce these policies at runtime, not after the fact. If an agent attempts a tool call outside its policy, the system should block it and generate an alert.
Separate pre-LLM and post-execution guardrails: Pre-LLM guardrails redact sensitive inputs and block prompt injection. Post-execution guardrails evaluate outputs for faithfulness, toxicity, and policy violations. Both layers must operate independently. Deploy them as enforceable runtime controls, not advisory scans.
Establish behavioral baselines: Instrument span-level telemetry across the full decision tree of agent calls, tool invocations, and sub-agent outputs. Use this telemetry to establish expected behavior patterns. Alert when an agent deviates: a compliance agent that suddenly makes 10x more external API calls than its historical average isn't operating within its expected envelope.
Audit your AI supply chain: Evaluate every component: base models, fine-tuning datasets, retrieval corpora, third-party APIs, framework dependencies. A single compromised embedding model or poisoned retrieval index can propagate risk across dozens of downstream agents. Map these dependencies and flag known vulnerabilities.
Do not rebrand infrastructure scanning as AI-SPM: If your current tooling can't inventory AI-specific assets or evaluate AI-specific risks (hallucination, prompt injection, tool-call misuse), it isn't AI-SPM. Ask your vendor whether the tool maintains an AI-BOM and performs continuous runtime evaluation. If the answer is no, you still have a coverage gap.
Posture management for AI agents isn't an extension of cloud security. It's a distinct discipline that operates at the semantic layer where agents make decisions. Your organization's ability to grant agents autonomy depends directly on your ability to oversee them in real time.
