Reports say covert testing found that Claude and ChatGPT generated viable proposals for terrorist attacks, and a senator is reportedly planning AI hearings. Details about the tests are thin. We don't know the prompts, the scoring criteria, who judged "viable," or whether anyone reran the tests.
That gap is the point of this checklist. A disturbing output is where assurance work starts. It doesn't end there. This checklist covers how to make a harmful-capability finding credible, decide what it means, tie it to controls, and communicate it without overclaiming or hand-waving.
Prerequisites
Before you work through the items, confirm you have:
- A defined harm taxonomy that names the capability categories you evaluate, for example, chemical, biological, cyber, and physical violence.
- An approved [Red Teaming](/glossary/Red Teaming) protocol with legal and security sign-off on handling hazardous outputs.
- A named owner who can authorize deployment changes.
- A secure repository for storing outputs. Don't paste them into tickets or chat.
The Checklist
1. Record the full test specification.
Done means the prompts, system prompts, model version, date, temperature and other sampling settings, tool access, and interface (API or consumer app) are all captured. Without these, a result is an anecdote.
Good looks like: a third party could rerun the test from your record alone.
2. Define "viable" before scoring.
Write the rubric first. Specify what separates a plausible-sounding narrative from an operationally specific, accurate, and actionable plan. Score dimensions separately: specificity, factual correctness, feasibility, and uplift beyond what a search engine provides.
Good looks like: a written rubric, dated before testing, with anchored examples at each score level.
3. Use qualified, independent graders.
A reviewer who knows the threat domain should judge feasibility. The team that built the prompts shouldn't be the only scorer.
Good looks like: at least two graders with documented domain expertise, plus recorded inter-rater agreement and a process for resolving disagreements.
4. Establish a baseline.
Compare model output against what a motivated person could get from public sources or a non-AI workflow. Uplift, not mere output, is what changes the risk picture.
Good looks like: a documented baseline condition, such as a search-only control, and a stated conclusion about marginal uplift.
5. Replicate the finding.
Run the test repeatedly across sessions, seeds, and prompt paraphrases. Report how often the behavior occurs, not just that it occurred once.
Good looks like: a reported frequency across a stated number of runs, with variation noted.
6. Distinguish a plausible proposal from a deployable capability.
A fluent plan isn't the same as a capability. Ask what a real actor would still need: materials, expertise, access, time, and evading detection. Record which barriers the model addressed and which it ignored.
Good looks like: a gap analysis listing the remaining real-world barriers, with a judgment on whether the output meaningfully lowers any of them.
7. Test under realistic conditions.
Include adversarial techniques such as jailbreaks, multi-turn escalation, and role-play framing, since these reflect how misuse actually occurs. Map them to MITRE ATLAS or the NIST AI 100-2 taxonomy so results are comparable.
Good looks like: tested attack classes are labeled against a recognized taxonomy, and untested classes are listed as known gaps.
8. Set thresholds before you see results.
Define in advance what finding triggers what response: continue with monitoring, restrict, escalate, or pause. Tie thresholds to your risk appetite under ISO/IEC 23894 or your NIST AI RMF implementation.
Good looks like: a threshold table approved by the risk owner before testing began.
9. Map the finding to a specific control.
Each confirmed finding should produce a decision: tighter refusal behavior, input and output filtering, access restrictions, usage policy changes, added monitoring, or further testing. If you rely on a model provider's safeguards, verify them rather than assuming them.
Good looks like: every finding has a linked control, an owner, a due date, and a retest plan.
10. Retest after remediation.
A fix that isn't retested is a hypothesis. Rerun the original specification plus new variants.
Good looks like: validation evidence showing before-and-after results under identical conditions.
11. Handle hazardous outputs safely.
Restrict who can view raw outputs, redact in reports, and follow Responsible Disclosure norms if the finding implicates a third party's model.
Good looks like: a documented handling procedure and an access log.
12. Document the decision trail.
Record the finding, the confidence level, the options considered, the decision, and who approved it. This is what a later reviewer, or a regulator, will ask for.
Good looks like: a decision record that stands alone, without the author in the room.
13. Write the uncertainty statement.
State what the evidence shows, what it doesn't, and what you'd do to learn more. Separate observed results from inference.
Good looks like: a plain-language paragraph with explicit confidence and named limitations, reviewed by legal and communications before release.
14. Prepare for external scrutiny.
When political or press attention arrives, you'll be asked for conclusions quickly. Pre-draft a short brief and a longer technical annex so you aren't improvising under pressure.
Good looks like: a two-tier summary, updated whenever new evidence arrives, with a single accountable spokesperson.
Common Mistakes
- Treating one output as proof. A single alarming response says little about frequency or severity.
- Scoring on tone. Confident, well-structured text reads as competent. Grade accuracy and actionability, not polish.
- Skipping the baseline. Without it, you can't tell uplift from restated public knowledge.
- Setting thresholds after the fact. Post hoc thresholds invite motivated reasoning in both directions.
- Minimizing or catastrophizing. Both distort the decision. Dismissing a reproducible finding is as indefensible as announcing capability you haven't demonstrated.
- Leaving findings without owners. A report no one is accountable for changes nothing.
Next Steps
Audit your last harmful-capability evaluation against items 1 through 5. If you can't reproduce it from your own records, fix the documentation before you expand the test set. Then pre-agree thresholds with your risk owner, and fold the process into your AI Management System under ISO/IEC 42001 so evaluation, response, and review run as a repeatable Plan-Do-Check-Act (PDCA) loop rather than a reaction to the next headline.





