When you're validating cost controls for an agentic AI deployment, capturing token consumption accurately is essential. This template provides a telemetry schema that separates token spend from value attribution, handling the complexity of real agent hierarchies.
Purpose of the Template
This schema is designed to capture per-span token data in agentic systems where a single user request triggers multiple LLM calls, tool invocations, and sub-agent runs. It's tailored for model risk teams needing to:
- Attribute token costs to specific agents, tenants, and outcomes.
- Ensure token spend aligns with business value.
- Detect cost drift between forecast and production.
- Support audit inquiries about AI operating expenses.
The schema accounts for three token types that bill differently: input tokens you send, output tokens the model returns, and reasoning tokens the model generates internally but doesn't show in the response. A framework that counts only visible tokens will understate actual costs.
Prerequisites
Before implementing this schema, ensure:
- Your observability stack can ingest structured JSON events at span granularity.
- You have access to the model provider's resolved pricing at request time.
- Your agent framework can tag spans with trace IDs linking sub-agent calls to the parent request.
- You can join gateway-side telemetry with agent-side telemetry.
If you're only metering tokens at the API gateway, you're missing sub-agent calls that bypass it. This schema assumes you instrument at the agent execution layer.
The Telemetry Schema
{
"span_id": "a3f9c2",
"trace_id": "req-8817",
"parent_span_id": "null",
"agent_id": "claims-support",
"tenant_id": "acct-4412",
"span_type": "llm_completion",
"model": "gpt-5.4",
"tokens": {
"input": 1840,
"output": 320,
"reasoning": 2100
},
"unit_price_usd": {
"input": 0.0000025,
"output": 0.000015
},
"cache_hit": false,
"resolved_at": "2026-07-10T14:03:11Z",
"outcome": "ticket_resolved",
"context_metadata": {
"feature": "auto-triage",
"user_segment": "enterprise",
"retry_count": 0
}
}
Field Definitions
span_id: Unique identifier for this unit of work. Links to downstream tool calls and sub-agent invocations.
trace_id: Groups all spans from a single user request. Your cost-per-run calculation sums across this ID.
parent_span_id: Links sub-agent calls back to the parent span. Set to null for top-level spans.
agent_id: The agent that generated this span. Use this to route costs to the team that owns the agent.
tenant_id: The customer or business unit this request serves. Required for multi-tenant cost allocation.
span_type: One of llm_completion, tool_invocation, retrieval, sub_agent, or evaluation. Evaluation spans capture LLM-as-a-judge overhead that most token models never itemize.
model: The model name as returned by the provider. Store this exactly as billed, since routing changes between billing cycles.
tokens.input: Tokens sent to the model in this span.
tokens.output: Tokens returned in the visible response.
tokens.reasoning: Tokens generated internally by reasoning models. These bill at the output rate but don't appear in the response. Ignoring this field understates cost.
unit_price_usd: The resolved price per token at request time. Capture this when the span executes, not when you generate reports, because provider pricing changes and a stale price rewrites your history.
cache_hit: Boolean indicating whether prompt caching reduced the effective cost. A cache hit changes the cost per token, and ignoring this overstates spend against the invoice.
resolved_at: ISO 8601 timestamp when the span completed. Use this to correlate cost with provider billing periods.
outcome: The business result this span contributed to, such as ticket_resolved, pr_merged, query_answered, or escalated_to_human. Pair this with cost to calculate value per token.
context_metadata: Optional nested object for feature flags, A/B test cohorts, retry counts, and user segments. Use this to slice cost by the dimensions your finance team asks about.
Customization Options
Financial Services
Add fields to support SR 11-7 model risk management requirements:
"validation_metadata": {
"model_version": "v2.3.1",
"approval_id": "MRM-2026-042",
"risk_tier": "tier_2"
}
This links token spend to the specific model version under validation and supports audit inquiries about which approved model generated the cost.
Multi-Model Routing
If your agent routes between models based on complexity or cost, add:
"routing_metadata": {
"original_model": "gpt-5.4",
"fallback_reason": "rate_limit",
"fallback_sequence": 1
}
This captures retry and fallback fan-out, where automatic retries multiply per-request token cost without changing the request count.
EU AI Act Compliance
For operations under the EU AI Act's transparency obligations, add:
"regulatory_metadata": {
"ai_system_id": "AIS-2026-089",
"risk_category": "limited_risk",
"disclosure_provided": true
}
This supports Technical Documentation (Annex IV) requirements by linking token consumption to the registered AI system.
Validation Steps
1. Verify Token Completeness
Run a spot check on ten production traces. For each trace, sum tokens across all spans and compare to the provider invoice line item for that request. If your sum is consistently lower, you're missing reasoning tokens or sub-agent calls.
2. Validate Price Resolution
Check that unit_price_usd matches the provider's billing rate on resolved_at. If you're storing list prices instead of resolved prices, your historical cost reports will be wrong whenever pricing changes.
3. Test Trace Completeness
Instrument a known agent with a fixed call pattern: one parent span, two tool calls, one sub-agent. Verify that trace_id groups all four spans and that parent_span_id correctly links children to parents. If tool calls appear orphaned, your gateway-side and agent-side telemetry aren't joined.
4. Calculate Cost Per Run
Implement this aggregation function and run it on a sample trace:
def run_cost(spans):
total = 0.0
for s in spans:
t = s["tokens"]
p = s["unit_price_usd"]
total += t["input"] * p["input"]
# reasoning tokens bill at the output rate
total += (t["output"] + t["reasoning"]) * p["output"]
# adjust for cache hits if your provider supports it
if s.get("cache_hit"):
total *= 0.1 # example discount; check your provider
return total
If your calculated cost per run doesn't reconcile to the provider invoice within 5%, you have a measurement gap.
5. Attribute Cost to Outcome
For one day of production traffic, group spans by trace_id, sum cost per trace, and join to outcome. Calculate the percentage of traces where outcome is null. If it's above 10%, your outcome instrumentation is incomplete and you can't measure value per token.
This schema won't tell you whether an expensive reasoning loop was worth it. That's a value question, not a telemetry question. What it will tell you is exactly which agent, which step, and which tenant generated the cost, which is the prerequisite for answering the value question later.
