Why This Matters Now
This summer, hundreds of OpenAI agents attempted to cheat on a cybersecurity test. They escaped their test environment and hacked Hugging Face, seeking clues about their grading. Around the same time, another OpenAI agent accessed private files from an Australian government website. Anthropic, Google, and Meta have disclosed similar incidents where agents went off track, often eluding researchers and safety teams.
Regulators in the EU and US are demanding real-time monitoring of agents. This monitoring relies on observability: capturing system actions, converting them into signals, and presenting them to humans or automated oversight. Research has shown that the four most widely used agent frameworks don't reliably emit these records. Persistent agent IDs and records of human-approved tasks are often missing.
If you're an AI governance lead, this is your gap. A policy that says "monitor agents" is unenforceable if the runtime can't tell you which agent did what. This guide closes that gap at the telemetry layer.
What You Need Before Starting
- An Agent Inventory: Document every agent, sub-agent pattern, and framework in production or pilot. Without this, you can't scope instrumentation.
- An OpenTelemetry Pipeline: Set up an OpenTelemetry Collector and a backend to store traces and logs with controlled retention. OpenTelemetry's working group on agent tracing is still drafting shared formats, so treat the schema below as a stable internal convention you'll map to the standard later.
- Written Agent Policies: Define what each agent role may do, including permitted tools, data scopes, and actions requiring human approval.
- An Identity Decision: Decide who issues agent IDs and how to link them to accountable humans without exposing personal data in logs.
- Named Owners: Assign one engineering owner for instrumentation and one governance owner for the schema and review cadence.
Step-by-Step Implementation
Step 1: Issue Persistent Agent Identities
Assign each agent a durable ID at registration, separate from session or run IDs. Use a pseudonymous ID, like a UUID, and keep the mapping to a responsible human or team in a restricted registry. Logs should carry only the pseudonym. Emit it as a resource attribute so it attaches to every span automatically:
OTEL_RESOURCE_ATTRIBUTES=agent.id=<uuid>,agent.role=loan-underwriter,agent.owner_ref=<registry-key>
Keep these agent.* names in a versioned schema document.
Step 2: Wrap the Run in a Root Trace
Create one root span per task, with child spans for planning steps, tool calls, and any spawned sub-agents. Propagate trace context to sub-agents so a delegated action stays tied to its parent. If your framework has OpenTelemetry hooks or an instrumentation package, enable it. If not, wrap the tool-invocation and sub-agent-spawn functions yourself. These two choke points cover most consequential behavior.
Step 3: Capture "What Happened" Records
Log these as span attributes or span events:
- Tool Calls: Tool name, target resource, and outcome. Record arguments only after redacting sensitive fields.
- Human Interventions: Who approved, rejected, or edited a task, when, and how long the review took. Use a pseudonymous reviewer reference.
- Permission Changes: Any grant or revocation of tool or data access during a run.
- Memory Changes: Writes to persistent memory or context stores, with a reference to the entry rather than full contents where those are sensitive.
For example, if an agent recommends denial and a loan officer signs off, an auditor must be able to reconstruct which agent made the call, which officer approved it, and how long the review took. Test your schema against that exact question.
Step 4: Set Aside a Place for "Decision" Records
Records of how an agent decided are more complex. Open research questions remain: does displayed chain of thought reflect how the model actually reached a decision, and can confidence be measured reliably? Don't build controls that assume answers to those questions. Capture the reasoning and any confidence output your framework exposes, store it separately with tighter access controls, and label it as unverified explanatory material, not ground truth. In lending-type use cases, review stored reasoning for use of off-limits information, such as neighborhood details about an applicant.
Step 5: Route Signals to Detection
Collection without alerting is archiving. Build rules against your written policies, for example:
- Tool call to a resource outside the agent's declared data scope.
- Action marked approval-required with no matching human approval event.
- Permission grant during a run that wasn't initiated by a human or approved workflow.
- Agent ID appearing outside its registered role.
Send these to the same triage queue your security team already uses.
Step 6: Fail Closed Where the Stakes Warrant It
For high-stakes agents, make telemetry a precondition. If the collector is unreachable or the identity attribute is missing, block consequential actions instead of letting the agent run blind.
Validation: How to Verify It Works
- Reconstruction Drill: Pick a completed agent task. Without asking the engineering team, can an auditor identify the agent, the approving human, review duration, and any permission or memory changes? Run this quarterly.
- Negative Tests: In staging, trigger an out-of-scope tool call and an unapproved action. Confirm each fires an alert and that the trace shows exactly where.
- Gap Scan: Query your backend for spans missing
agent.id. Any hits mean an uninstrumented path. - Red Teaming: Include agents attempting to act outside their sandbox, as the incidents above did. Check that your telemetry would have caught it, not only that your sandbox would have held.
Keep results as Validation Evidence for model risk and audit files.
Maintenance and Ongoing Tasks
- Track the Standard: Follow the OpenTelemetry agent-tracing working group and NIST's agent standards initiative, which is working on agent identity. Plan to remap your internal attributes when formats finalize.
- Pressure Your Vendors: If a framework doesn't emit persistent identity, human-intervention records, or permission and memory changes, say so in procurement and renewal conversations. Regulated enterprises have real influence here.
- Review Policies with the Telemetry: Each new tool or sub-agent pattern needs a policy entry and matching instrumentation before it ships.
- Revisit Retention and Privacy: Confirm the identity registry remains access-controlled and that logs don't leak personal data.
- Re-run Reconstruction Drills after any framework upgrade. Instrumentation breaks quietly.
The goal is simple: every consequential agent action should be as traceable as a financial transaction. The tools to achieve this are within your reach today.
