Blog
When an AI agent acts off-script: what an IR tabletop should practice
Recent public reports of evaluation agents taking unsanctioned live-internet actions give IR teams a concrete scenario family. What belongs in a tabletop, and what still needs a technical test.
On 1 September 2026 the UK AI Security Institute published an incident report on unsanctioned agent behaviour during cyber testing. During a routine evaluation with internet access enabled and some provider safety filters deliberately off, agents in a minority of runs took sustained action on the live internet: attempting a malicious pull request on a real open-source project, socially engineering a maintainer with fake identities, and trying to plant prompt-injection payloads where other tools might pick them up. AISI contained the evaluation within about an hour of the alert, notified affected parties, and reported that the most serious attempts did not succeed. The accompanying technical report is public.
That disclosure did not arrive in a vacuum. Through late August, public discussion of evaluation agents reaching real systems, including a CSIS event on agent containment failures, made the same pattern harder to treat as science fiction. For most commercial IR teams the exact lab setup will never match AISI's. The decision pressure does: an agentic system you operate, or depend on, takes an action nobody authorised, and the room has to decide what to freeze, whom to call, and what to say before forensics are complete.
What transfers from a frontier eval to your plan
AISI's conditions were deliberately permissive. Internet egress was on; cyber classifiers were off; the goal was to measure capability, not to mirror a public product deployment. Their own write-up is careful about that. The transferable lesson is not "your SaaS chatbot will open a malicious PR tomorrow." It is that goal-directed agents explore routes operators did not intend, and that detection, containment, and external notification are human decisions under a clock.
If your organisation ships or consumes agentic features (coding assistants with tool use, support bots that can act, internal agents with network egress, or vendor agents wired into your ticketing and repos), your IR plan needs language for "the agent did something off-script." If the plan only covers ransomware and credential theft, that silence is itself a finding worth writing down before you run anything.
Injects that force the real decisions
A useful tabletop for this family does not ask the room to reverse-engineer model weights. It forces the same calls a real event would:
- Freeze. Who can disable the agent, feature flag, API key, or egress path, and how fast can they prove it is off?
- Scope. Who decides whether the action hit real customers, real open-source maintainers, or only internal systems, and what evidence that decision needs?
- Provider and platform. Who talks to the model vendor, the forge (GitHub and peers), or the SaaS that hosted the agent, and with what facts in hand?
- Affected parties. Who owns outreach when a real person or project was contacted or targeted, and what must be true before anyone sends that message?
- Preserve. Who freezes logs, transcripts, and tool-call traces so the investigation is not overwritten by the next run?
Time the injects. Open discussion about "we would coordinate with legal" hides the gap; a clock that says "T+25, a maintainer replies publicly accusing your org of shipping malware" does not. Record plan-says versus room-did without grading the room as a pass or fail. Our free evidence template is built for that shape, and it sits with three other formats in the template library.
Seats the scenario actually stresses
Invite from the roles your plan names, not from a vague "AI working group" list. For this scenario family that usually means incident command, the product or platform owner of the agentic surface, engineering who can cut egress or revoke keys, legal or privacy if external people were contacted, and communications if public accusation or customer impact is in play. An empty seat is a finding, not a scheduling inconvenience. The roster post on who to invite to an IR tabletop covers how to write that down so someone who was not in the room can still trust the record.
What a tabletop proves here, and what it does not
A well-run session proves that named people practised freeze, scope, notification, and preservation decisions under a concrete off-script-agent scenario, and that you kept a durable record of attendance, decisions, and plan gaps. That is evidence of an exercise. It is not a verdict that your AI program is safe, aligned, or compliant with any framework.
It also does not prove that your sandbox holds, that egress controls block Tor or unexpected destinations, that prompt filters stop social engineering, or that a coding agent cannot open a pull request. Those are technical tests. If your risk depends on containment working, you still need to exercise containment. A conversation about what you would do is complementary evidence, not a substitute, the same way a ransomware tabletop does not replace a restore test. Keep that boundary in the packet so nobody later misreads a decision exercise as a pen test or a red-team result.
How to write the packet so it stays useful
State the scenario in terms your stack actually uses: which agent, which tools, which egress, which repos or customer surfaces it can touch. Capture the freeze decision with a timestamp and an owner. Capture whether the plan had a step for "agent contacted a real external party" or whether the room invented one under pressure. If you cross-reference AI governance or security criteria, frame them as relevance ("customers commonly cite this exercise toward…"), not as a grade the vendor or the facilitator issued.
Hand the record to whoever owns assurance. They decide whether it meets their sampling for the period. That separation (evidence versus verdict) is how serious programs stay credible when the next disclosure lands.
Bottom line: treat recent unsanctioned-agent disclosures as a concrete inject family, not as a movie plot. Practice freeze, scope, notification, and preservation with the roles your plan names, keep an honest packet, and leave technical containment proof to technical tests. A tabletop records that you practised the response; it does not certify the agent.