The Challenge
Your fintech engineering team deployed Cursor and GitHub Copilot to 400 developers. You published an AI coding tool security policy prohibiting customer data in prompts, secrets in generated code, and requiring review of all model outputs. Every engineer read it. The policy lived in a wiki.
Then an agent broke it without anyone noticing.
A developer asked Copilot to debug a failing payment reconciliation job. The agent called an internal Model Context Protocol (MCP) tool to fetch a sample record, which included a real customer's name, account number, and balance. This data entered the model's context, traveled in a prompt to an external provider, and was logged by a third-party service.
The wiki policy was never consulted. Nothing inspected what the agent sent or received. The failure occurred in real time, on a path no static control monitored.
The Environment and Constraints
Your organization deployed a typical control stack:
- IDE privacy settings with telemetry and training opt-outs configured per client
- Content exclusion rules with file and path patterns the assistant should ignore
- Rules files containing in-repo instructions to steer agent behavior
- Network allowlists limiting which endpoints machines could reach
- CI/CD scanning with static application security testing (SAST) and software composition analysis (SCA) checks running on commits
Each control helped, but none enforced policy when it mattered.
Configuration depends on correct setup on every machine, and it drifts the moment one developer skips a step. Rules files and IDE settings are advisory, a developer can ignore them, override them, or misconfigure them, and the agent keeps working. According to your team's research, 80% of developers admit to bypassing security policies when using AI coding tools.
Pre-merge scanning runs too late. By the time a SAST job flags a secret in a diff, the prompt that exposed a customer record has already reached an external model and been logged. Scanning catches the artifact but not the exposure.
Your team faced a second constraint most organizations miss: the inbound direction. Almost every AI coding control was built to stop data from leaving. The harder direction is data coming in. Coding agents pull context through MCP servers, WebFetch calls, and tool endpoints. Those endpoints can return personally identifiable information or protected health information directly into the agent's context. The agent didn't ask for a customer's medical record; it asked a tool for a record, and the tool handed one back.
The Approach Taken
Your team moved enforcement onto the request and response path itself. They identified three layers where controls can sit:
- IDE and client configuration (per-machine, advisory)
- The LLM and MCP gateway (the request and response path every prompt and tool call traverses)
- CI/CD pre-merge (post-hoc)
Only the middle layer sees every request and response. If a control lives here, no developer can opt out and no exposure slips past before a scan runs.
They implemented two categories of runtime guardrails:
Pre-LLM guardrails intercept inputs before they reach the model. They inspect the prompt and any tool payloads on the way out.
Post-execution guardrails inspect outputs before they're returned to the developer or acted upon. They examine generated code and tool responses on the way back.
Each guardrail produces one of three verdicts:
- Allow: the request or response passes unchanged
- Redact: sensitive content is removed while the request still completes (the default for secrets and PII, because it preserves developer workflow)
- Block: the full request is rejected (reserved for cases where the request itself is the threat, such as a prompt injection attempt)
The distinction between redact and block matters in practice. Redaction handles the common case, a secret or customer field slipping into context, without breaking the task. Blocking handles the adversarial case, where completing the request at all would be unsafe.
Your team deployed the Fiddler AI Control Plane to deliver inline enforcement on the agent's request and response path. It integrated with your existing LLM and MCP gateway without requiring new infrastructure, SDKs, or agent rewrites.
Results and Metrics
Fiddler's guardrails run in-environment and return verdicts in under 100ms. Models now process only approved inputs, and developers receive only approved outputs.
The inline enforcement created an immediate secondary benefit: every verdict became a telemetry event. The same check that redacts a secret records that it happened, against which developer, on which tool, with which model. Policy and visibility became one system instead of two.
Your team gained fleet-level intelligence they couldn't produce with per-machine tools: cost per pull request, adoption measured against seats, and model usage across every developer. Span-level telemetry rolled up to aggregate insights across the agentic hierarchy, surfacing fleet trends without hand-stitching individual tool calls.
What They Would Do Differently
Your team identified four failure modes that accounted for most surprises during rollout:
Per-machine drift: IDE-level exclusions configured on some seats but not others created silent coverage holes that no dashboard reported. They should have assumed configuration drift from day one.
Redaction placement: Their initial implementation redacted only on the way out, missing PII entering through tool responses. Placing guardrails on both paths closed the gap.
Over-blocking: Blanket blocking broke developer workflow and trained engineers to route around the control. Preferring redaction so requests still complete preserved adoption.
Untracked agents: Third-party and coding agents deployed without registration escaped policy entirely, because a control can't enforce against a workload it doesn't know exists. They now require registration before deployment.
Takeaways for Your Team
Start with a mapping exercise. Take your current controls and sort them into the three enforcement layers. Find where nothing sits on the runtime path.
If you're relying on wiki policies, IDE settings, or pre-merge scanning alone, you're enforcing after exposure has already occurred. Configuration is advisory. Scanning is retrospective. Only runtime enforcement intercepts the request before data leaves and the response before data enters.
Don't ignore the inbound direction. Most controls watch prompts and completions going out. They don't inspect responses coming back into the agent, which is exactly where inbound exposure originates. A post-execution guardrail on tool responses can redact sensitive fields before they ever enter context.
Prefer redaction over blocking. Redaction handles the common case without breaking workflow. Reserve blocking for adversarial inputs where completing the request itself would be unsafe.
Treat every verdict as telemetry. Policy and visibility should be one system. The same inline check that enforces policy creates a record of trust for every action, every decision, and every output.
The goal isn't just to monitor coding agents. It's to evaluate, control, and govern them, across first-party agents your teams build, third-party agents you deploy, and coding agents running across your developer organization.
