Your AI agent passed today's security scan. It handled adversarial prompts without incident. Your validation team signed off. Three weeks later, that same agent executes a compromised instruction it "remembered" from a poisoned interaction you cleared as safe.
This isn't a theoretical gap. Research from the University of Calgary examined 2,614 simulated multi-step attack trajectories and found that memory poisoning attacks can remain dormant through multiple normal interactions before activating. The problem: your security testing likely stops at the interaction boundary.
What This Checklist Covers
This checklist addresses trajectory-aware security testing for AI agents with persistent memory capabilities. It's designed for teams validating conversational AI, autonomous agents, or any system that carries context across sessions. If your AI can "remember" previous interactions, you need testing that spans multiple steps.
Prerequisites
Before using this checklist, confirm:
- Your AI system maintains persistent memory. This includes session history, user preferences, or learned context that carries forward. Single-turn chatbots without memory don't face this specific risk.
- You have access to the agent's memory architecture. You need visibility into what gets stored, how it's retrieved, and when it influences decisions.
- You can simulate multi-step interactions. Your test environment must support sequences of at least 5-10 interactions, not just isolated prompts.
- You've documented normal behavior baselines. You can't detect drift without knowing what stable operation looks like across multiple steps.
Checklist Items
1. Map memory persistence points
Identify every location where your agent stores retrievable information: conversation logs, user profile updates, task completion records, or learned preferences.
✓ Done when: You have a complete inventory of memory stores with retention periods, retrieval triggers, and access controls documented. A diagram should show what information flows from interaction N to interaction N+3.
2. Define trajectory test scenarios
Create test sequences of 8-12 interactions where earlier steps could influence later behavior. The University of Calgary research studied four attack types: chain poisoning, policy rewriting, backdoor triggering, and slow drift. Your scenarios should cover similar patterns in your domain.
✓ Done when: You have written test scripts where success or failure can only be determined by examining the full sequence, not individual steps. A scenario should show step 3 planting information that triggers detection at step 9.
3. Establish non-monotonic risk detection
Configure your security monitoring to flag cases where risk scores don't follow a simple increasing pattern. An agent might appear safer at step 5 than step 3, then show compromise at step 8.
✓ Done when: Your evaluation framework can detect U-shaped or W-shaped risk trajectories, not just linear escalation. Alerts should fire when an agent's behavior becomes less suspicious mid-sequence after an initial red flag.
4. Test memory retrieval under adversarial conditions
Inject misleading information into memory stores, then run normal task sequences. Don't just check whether the agent rejects bad input, verify it doesn't later retrieve and act on poisoned memories during legitimate operations.
✓ Done when: You've confirmed that adversarial content inserted at T=0 doesn't influence decisions at T=5 or T=10. Test logs should show the agent consulted its memory but correctly ignored or quarantined the planted information.
5. Validate memory isolation between users/sessions
If your agent serves multiple users, confirm that poisoned memory in User A's session can't leak into User B's context or into the base model's behavior.
✓ Done when: Cross-contamination tests show zero information leakage across session boundaries. Cryptographic separation or namespace isolation should survive Red Teaming attempts to breach it.
6. Implement trajectory-aware audit logs
Your logs must link interactions across time. A single-step view won't reveal that today's decision was influenced by last week's poisoned input.
✓ Done when: You can reconstruct the complete decision chain for any agent action, tracing which memories were retrieved and when they were originally stored. A query should show "Action X at timestamp T drew on memory Y from session Z."
7. Set memory refresh and validation intervals
Determine how long information persists before requiring revalidation. Indefinite memory retention amplifies poisoning risk.
✓ Done when: Every stored memory has an expiration policy and revalidation trigger. Automated workflows should flag memories older than 30 days for human review before reuse.
8. Test tool-use decisions across trajectories
If your agent can invoke APIs, access databases, or execute commands, verify that poisoned memories can't manipulate tool selection or parameters in later steps.
✓ Done when: Multi-step tests confirm that compromised context from step 2 doesn't cause the agent to misuse tools at step 7. Tool invocation logs should show decision rationale traced back to validated memory sources only.
Common Mistakes
Testing only at the prompt boundary. You scan inputs for adversarial content but don't check what the agent remembers afterward. Memory poisoning succeeds after the prompt passes your filter.
Assuming linear attack progression. You expect compromised agents to behave obviously wrong immediately. The research shows attacks can remain dormant through several normal interactions.
Validating memory writes but not memory reads. You verify what goes into storage but don't test how retrieval influences downstream decisions.
Using single-interaction security benchmarks. Standard adversarial robustness tests evaluate one prompt at a time. They can't detect attacks that span multiple steps.
Treating all memory equally. You apply uniform retention policies without distinguishing between low-risk preferences and high-risk operational instructions.
Next Steps
Start with item 2: build one trajectory test scenario for your highest-risk use case. If your agent handles customer support, create a sequence where step 3 contains misleading policy information and step 9 requires a policy decision. Run it. If your current testing would have missed the connection, you've confirmed the gap.
Then work backward through items 1 and 6 to build the visibility you need. You can't test trajectories without mapping memory flows and linking logs across time.
Don't wait for a production incident to discover that your security testing stops where your agent's memory begins.
