AI incident response plans are what separate a contained embarrassment from a regulatory investigation. The model that starts inventing refund amounts, the autonomous AI agents that quietly email six hundred customers the wrong renewal date, the classifier that begins rejecting every application from one postcode — none of these trip an alarm in a normal monitoring stack. They look like healthy traffic all the way down.
That is the uncomfortable part. Conventional IT incidents announce themselves: something is down, an error rate spikes, a pager fires. AI systems fail while returning HTTP 200 in ninety milliseconds. By the time a person notices, the failure has been running for days and has touched every record it was permitted to touch. An AI incident response plan exists to shorten that window and to make the hour that follows deliberate rather than improvised.
Most organisations already own a cyber incident playbook, and most of it still applies. What does not apply is the assumption that a fault is binary, reproducible, and fixable with a patch. A model can be wrong for one customer segment and right for everyone else, and it can be wrong today with the same weights that were right last week. Your AI incident response plan has to survive that ambiguity without stalling.
This guide covers what actually counts as an AI incident, the six phases to build an AI incident response plan around, a severity matrix you can copy, the containment options ranked by how much damage they stop, and how to rehearse the whole thing before a real failure forces you to learn it live.
Table of contents
- Why an AI Incident Response Plan Differs From Your Cyber Playbook
- What Counts as an AI Incident, and What Does Not
- The Six Phases of an AI Incident Response Plan
- Severity Levels: Sizing an AI Failure in the First Ten Minutes
- Detection and Triage: Knowing the Model Has Failed
- Containment: Rollback, Degrade, or Pull the Kill Switch
- Root Cause Analysis When the System Is Non-Deterministic
- Communication, Disclosure and the Regulatory Clock
- Roles and Runbooks Your AI Incident Response Plan Needs
- Rehearsal: Testing the AI Incident Response Plan Before You Need It
- Frequently Asked Questions About an AI Incident Response Plan
- Your First AI Incident Response Plan in Thirty Days
- References
Why an AI Incident Response Plan Differs From Your Cyber Playbook
A ransomware runbook assumes an adversary, a compromised asset, and a clear moment of detection. An AI failure often has none of those. The system is behaving exactly as engineered — it is the engineering that is wrong, or the data has shifted underneath it. That single difference reshapes every phase of an AI incident response plan.
The failure is statistical, not binary
Traditional services are up or down. A model is 94% right, then 91%, then 78% for one language and 96% for another. There is rarely a moment when it “breaks”, so an AI incident response plan needs a declared threshold — a number agreed in advance at which degraded quality becomes an incident. Without that number, every conversation becomes a debate about whether this is really a problem yet.
There is no patch, only a rollback or a retrain
You cannot hotfix a model the way you hotfix a null-pointer. The realistic options are reverting to a previous version, tightening a prompt or a guardrail, narrowing the scope of what the system is allowed to do, or turning it off. All four are decisions with business consequences, which is why an AI incident response plan has to name who is allowed to make them at three in the morning.
The blast radius is measured in decisions, not systems
A conventional outage affects availability. An AI failure affects outcomes — records changed, claims declined, prices quoted, messages sent. Containment therefore means answering “what did it already do, and to whom?” before it means “how do we restart it?” That question is only answerable if you logged inputs, outputs and tool calls in the first place.
Evidence is ephemeral unless you capture it
Prompts, retrieved context, model version, temperature, tool arguments and intermediate reasoning all shape the output, and none of them survive in a standard application log. If your AI incident response plan does not specify what to capture at inference time, your investigation will consist of guesswork and screenshots from an angry customer.
| Dimension | Conventional IT incident | AI incident |
|---|---|---|
| Signal | Errors, latency, alarms | Silent wrong answers at normal speed |
| Detection source | Monitoring and synthetic checks | Sampled scoring, complaints, downstream audit |
| Reproducibility | Deterministic, usually repeatable | Probabilistic, may not repeat on rerun |
| Fix | Patch, restart, failover | Rollback, guardrail, scope cut, retrain |
| Blast radius | Systems and availability | Decisions, records and people |
| Evidence | Application and system logs | Prompts, context, versions, tool calls |
| Recovery test | Service responds again | Quality restored on a scored evaluation set |
What Counts as an AI Incident, and What Does Not
Half the value of writing the definitions down is that it stops the argument happening during the incident. A good AI incident response plan states plainly which events are in scope, so nobody has to interpret the policy while a model is misbehaving in production.
Four categories worth naming
An AI incident response plan should recognise four. Behaviour incidents, where the output is wrong, biased or unsafe. Security incidents, where someone manipulates the system through prompt injection, tool abuse or model extraction. Data incidents, where personal or confidential information leaks into a prompt, a log or an answer. Availability and cost incidents, where the system loops, floods a dependency or burns a month of budget overnight.
Model behaviour incidents
This is the category teams under-report because it feels like a quality issue rather than an incident. Systematic wrongness affecting a class of users is an incident, and an AI incident response plan should say so explicitly. One odd answer is a bug report. Two hundred odd answers with a shared pattern is an event with a timeline, an owner and a post-incident review.
Security incidents against the AI layer
Prompt injection through retrieved content, an agent persuaded to call a tool it should never touch, a jailbreak that unlocks restricted advice — these are attacks even though nothing in the infrastructure was breached. The OWASP Top 10 for LLM applications is the most practical checklist for enumerating them before you write the runbooks. Treat them as security events in your AI incident response plan, with the same escalation path as any other intrusion.
Data and privacy incidents
A model that reproduces training data, a support agent that pastes one customer’s details into another’s reply, a retrieval index containing documents the requester was never cleared to see. These carry statutory reporting duties in most jurisdictions, so your AI incident response plan should route them to the same people who handle any other personal-data breach — immediately, not after the technical fix.
The near miss deserves a ticket too
A guardrail that blocked something alarming is the cheapest intelligence you will ever get. Log near misses, review them monthly, and let them drive the test cases in your AI risk assessment work. Organisations that only record realised harm learn the same lesson twice.
The Six Phases of an AI Incident Response Plan
The phase structure below deliberately mirrors mainstream incident handling so it slots into what your team already does. The NIST guidance on incident response maps the same activities onto the Cybersecurity Framework functions, and reusing that shape keeps your AI incident response plan auditable rather than exotic.
Phase 1 — Prepare
Inventory every AI system in production and its owner. Define severity. Write the kill-switch procedure and test it. Agree logging requirements. Set up the channel where incidents get declared. Preparation is where 80% of the value of an AI incident response plan is created, and it is the only phase you can complete calmly.
Phase 2 — Detect
Combine automated quality signals, sampled human scoring, complaint routing and cost anomaly alerts. Detection is the weakest link in nearly every AI incident response plan we review, because teams instrument infrastructure thoroughly and outcomes barely at all. Continuous AI agent monitoring is what turns a six-week discovery into a six-hour one.
Phase 3 — Triage and classify
Confirm it is real, assign severity, name an incident lead, and start the log. Timebox this to ten minutes. The most common triage mistake is spending an hour proving the failure is genuine while it continues to run.
Phase 4 — Contain
Stop the bleeding using the least destructive option that works: narrow the scope, force human review, degrade to a simpler path, or turn the system off. Containment decisions belong to a named role, and your AI incident response plan should pre-authorise them so nobody waits for a director to wake up.
Phase 5 — Eradicate and recover
Find the cause, fix it, and prove the fix on a scored evaluation set before restoring full autonomy. Recovery in an AI incident response plan is not “the service is back”; it is “quality is back and we can demonstrate it”. Restore in stages, with a small traffic percentage first.
Phase 6 — Learn
Run a blameless review within five working days. Convert every finding into a test case, a monitor or a control. An AI incident response plan that does not feed its own evidence back into the evaluation suite will meet the same failure again with the same surprise.
Severity Levels: Sizing an AI Failure in the First Ten Minutes
Severity is the single most useful artefact in an AI incident response plan, because it converts a vague sense of alarm into an agreed response. Publish it, print it, and make it the first page anyone reads.
The three questions that set severity
Is the output reversible? How many people or records are affected? Does it touch money, safety, health, employment or a protected characteristic? Two yeses in the wrong places should escalate immediately, regardless of how confident anyone feels about the underlying model.
Reversibility beats visibility
Teams instinctively rank incidents by how embarrassing they are. Rank them instead by how hard the damage is to undo. A chatbot saying something silly on a public channel is highly visible and trivially reversible. An agent that has been silently amending ledger entries for nine days is invisible and close to irreversible.
Publish the matrix before you need it
The matrix below is a starting point to argue with, not a standard to adopt unchanged. Adjust the thresholds to your risk appetite, then wire the response times into your on-call rota. A severity matrix that nobody is paged against is decoration, and an AI incident response plan built on one is theatre.
| Level | Definition | Example | Response | Who is paged |
|---|---|---|---|---|
| SEV1 | Irreversible harm, safety, or regulated decisions at scale | Agent issued incorrect clinical or financial guidance to customers | Contain within 30 minutes, executive briefed within 1 hour | Incident lead, exec sponsor, legal, DPO |
| SEV2 | Systematic wrong outputs affecting a customer segment | Pricing model quoting 20% low for one region | Contain within 2 hours, review within 48 hours | Incident lead, product owner, engineering |
| SEV3 | Degraded quality, contained scope, reversible | Summaries omitting a required disclaimer | Next business day, fix in current sprint | Product owner and engineering |
| SEV4 | Near miss or single anomaly, no realised harm | Guardrail blocked an out-of-scope tool call | Log and review at monthly quality meeting | Owning team only |
Detection and Triage: Knowing the Model Has Failed
You cannot respond to what you never see, and the average AI failure is discovered by a customer rather than a dashboard. Fixing that imbalance is the highest-return investment available when you build an AI incident response plan.
Instrument outcomes, not just uptime
Availability, latency and error rate tell you the pipes are open. Add outcome signals: task success on a held-out set, groundedness against retrieved context, refusal rate, tool-selection accuracy, and the rate of human overrides. A rising override rate is often the earliest honest signal that something has drifted. The measurement side is covered in depth in our guide to AI agent evaluation metrics.
Sampled scoring in production
Score a random sample of live interactions continuously — a hundred a day is enough to catch a step change in most workflows. Use a judge model for breadth and a human reviewer for the disputed cases. Sampling is cheap, and it is the only detection method in an AI incident response plan that finds failures nobody has thought to write a rule for.
Make the human report channel obvious
Frontline staff notice AI failures long before telemetry does, and they usually tell nobody because there is no route. Put a one-click “this answer was wrong” control in the interface, route it somewhere staffed, and treat clusters as an alert. Every mature AI incident response plan we have seen has a fast, low-friction human channel at its front door.
Set alert thresholds you will actually action
An alert that fires daily and is dismissed daily is worse than no alert, because it trains the team to ignore the category. Set thresholds against a rolling baseline, alert on sustained change rather than single points, and route cost anomalies to the same channel — a runaway loop is an incident, not a billing surprise.
Containment: Rollback, Degrade, or Pull the Kill Switch
Containment is the phase where an AI incident response plan earns its cost. The goal is not elegance — it is stopping further wrong decisions with the smallest acceptable loss of service, while preserving the ability to investigate.
Know what “off” means before the incident
“Turn it off” sounds simple until someone asks whether that means the endpoint, the agent’s scheduled jobs, its queue consumers, its API keys, or the downstream automations that consume its output. Write the exact sequence into the AI incident response plan, name the person who can execute it without approval, and test it quarterly. An untested kill switch is a hypothesis.
Graceful degradation is usually the right first move
Full shutdown transfers the entire workload to humans who are not staffed for it, which is its own incident. Better intermediate steps exist: route to a deterministic fallback, restrict the agent to read-only tools, force human approval on every action, reduce autonomy to suggestion-only, or limit the system to the segments still known to be safe.
Revoke the agent’s credentials, not just its traffic
Agents act through tokens, service accounts and integrations, and blocking inbound traffic does nothing about a job already running with a valid credential. Containment for autonomous agents means revoking or scoping down what the identity can do, then confirming no queued work is still executing under the old permissions.
Preserve evidence while you contain
Snapshot the model version, prompt templates, retrieval index state, configuration and the full trace log before you change anything. Teams routinely roll back first and lose the evidence needed to explain what happened, which turns the post-incident review into speculation and leaves the regulator’s questions unanswerable. Naming the snapshot list in the AI incident response plan is what makes this happen under pressure.
| Option | Stops the harm | Service impact | Time to execute | Use when |
|---|---|---|---|---|
| Full kill switch | Completely | Severe | Minutes | Harm is irreversible or safety-related |
| Read-only mode | Stops new changes | Moderate | Minutes | Agent is writing to systems of record |
| Human-in-the-loop gate | Mostly | Slower service | Under an hour | Volume is low enough for review |
| Version rollback | If the change caused it | Minimal | Minutes | A recent deploy correlates with onset |
| Scope restriction | For affected segments | Partial | Under an hour | Failure is isolated to known cohorts |
| Deterministic fallback | Completely | Reduced capability | Minutes | A rules-based path already exists |
Root Cause Analysis When the System Is Non-Deterministic
Investigation is where AI incidents diverge hardest from ordinary engineering. The same input can produce a different output on rerun, so “I could not reproduce it” is not evidence of anything.
Reproduce with the exact same inputs and versions
Pin the model version, temperature, system prompt, retrieved documents and tool responses, then replay. If the failure reproduces, you have a defect. If it reproduces one time in twenty, you have a probabilistic failure that will recur at scale, and the AI incident response plan should push the fix towards a guardrail rather than a prompt tweak.
Work the pipeline backwards
Check the output, then the reasoning trace, then the tool calls, then the retrieved context, then the query, then the input. Most root causes sit in retrieval and data long before they sit in the model, which is why teams with strong data management practices resolve AI incidents markedly faster.
Distinguish a defect from a drift
A defect appeared with a change you made. A drift appeared because the world changed — new product names, seasonal language, a supplier’s document format, a customer base that shifted. Defects get rolled back. Drift gets a monitor, a refreshed evaluation set and a retraining schedule, and your AI incident response plan should treat them as different workflows with different owners.
Write the finding as a testable case
Every root cause should leave behind a case in the evaluation suite that fails on the old version and passes on the new one. That single discipline is what stops the same incident recurring in six months, and it converts incident work into permanent capability rather than firefighting.
Communication, Disclosure and the Regulatory Clock
Technical containment is half the job. The other half is what you tell customers, staff and regulators, and how quickly. Getting this wrong converts a manageable failure into a trust problem that outlasts the fix by years.
Tell affected users before they tell each other
Say what happened, what it means for them, what you have already done, and when they will hear from you next. Do not promise a root cause you do not have. A short honest update at hour two is worth more than a polished statement on day four, and an AI incident response plan that pre-drafts those templates makes hour two achievable.
Know which clocks start when
Under the EU AI Act, providers of high-risk systems must report serious incidents to the relevant market surveillance authority — in general no later than fifteen days after becoming aware, with much tighter deadlines for death or serious disruption to critical infrastructure. Personal-data breaches carry their own separate seventy-two-hour duty. Your AI incident response plan should list every applicable clock next to the person who starts it.
Keep one written record
One timeline, one channel, one document, updated as events happen rather than reconstructed afterwards. Record decisions and who made them, not only actions. Regulators and insurers both ask the same question — what did you know, and when — and a contemporaneous log is the only answer that holds up.
Brief the frontline first
Support and sales will field the questions long before any statement is published. Give them a short internal note with the facts they may share, the ones they may not, and where to escalate. Frontline staff improvising an explanation is one of the most common ways an incident gets worse, and it is entirely preventable.
Roles and Runbooks Your AI Incident Response Plan Needs
Structure beats heroics. The organisations that handle AI failures calmly are not the ones with the cleverest engineers; they are the ones where five people knew their job before the pager went off.
The five roles worth naming
An incident lead who coordinates and is explicitly not debugging. A technical investigator. A business decision-maker who can authorise turning revenue-generating systems off. A communications owner. A scribe keeping the timeline. On a small team one person may hold two roles, but never the lead and the investigator.
One runbook per failure mode
Generic plans fail under pressure, so an AI incident response plan should carry a set of short, specific runbooks: hallucinated output in customer-facing content, prompt injection through retrieved documents, agent taking unauthorised actions, personal data appearing in outputs, runaway cost or looping, model provider outage or deprecation. Each should fit on one page and name the first three commands or clicks.
Decision rights, written down
The most expensive minutes in any AI incident are spent finding someone who can approve the containment. Pre-authorise: the on-call engineer may degrade or restrict any system without approval; the incident lead may invoke a full shutdown for SEV1 and SEV2. Fold these authorities into your IT governance framework so they survive personnel changes.
| Role | Owns | First action | Does not do |
|---|---|---|---|
| Incident lead | Severity, sequencing, escalation | Declare, assign roles, open the log | Hands-on debugging |
| Technical investigator | Diagnosis and the fix | Snapshot evidence, replay the failure | Talk to customers |
| Business decision-maker | Trade-offs and shutdown approval | Confirm acceptable service loss | Direct the technical work |
| Communications owner | Internal and external messaging | Brief the frontline within the hour | Speculate on cause |
| Scribe | Timeline and decision record | Timestamp every decision | Take on remediation tasks |
Rehearsal: Testing the AI Incident Response Plan Before You Need It
A plan nobody has practised is a document, not a capability. Rehearsal is also the only reliable way to discover that the kill switch was decommissioned last quarter and nobody noticed.
Run the tabletop with the real people
Put the actual on-call engineer, product owner and communications lead in a room for ninety minutes with a realistic scenario: your support agent has been quoting a discontinued warranty policy for eleven days. Run it in real time. The same format works for cyber scenarios, and our guide to running a cyber tabletop exercise covers the facilitation mechanics in detail.
Inject failures into a staging agent
Go beyond discussion. Deliberately corrupt a retrieval index, revoke a tool credential mid-run, feed a prompt-injection payload through a document, or point the agent at a stale model version. Measure how long detection actually takes. Teams are consistently shocked by the gap between the detection time their AI incident response plan assumes and the one they measure.
Measure the rehearsal, not the vibe
Record time to detect, time to declare, time to contain and time to first customer communication. Track them across exercises. Those four numbers are the honest performance indicators for an AI incident response plan, and improving them is a more useful goal than adding pages to the document.
Frequently Asked Questions About an AI Incident Response Plan
Do we need a separate plan or an annex to the existing one?
An annex, in almost every case. Reuse your existing severity language, on-call rota, communication templates and review process. What genuinely needs writing fresh is the AI-specific detection, the containment options, the evidence list and the runbooks per failure mode. A standalone document tends to be forgotten precisely when it is needed.
Who should be able to declare an AI incident?
Anyone. A good AI incident response plan makes declaration cheap and de-escalation easy, because the alternative is a culture where people wait for certainty while a model keeps making decisions. The incident lead sets severity after the fact; the person who noticed should never have to justify raising it.
How fast should we be able to turn a model off?
Under fifteen minutes for anything customer-facing, and that number should be measured rather than assumed. Test it quarterly, from the pager to confirmed stop, including the downstream automations that consume the output. Most teams discover their true figure is hours the first time they check.
Do we have to report AI failures to a regulator?
It depends on the system and the jurisdiction. High-risk systems under the EU AI Act carry explicit serious-incident duties, personal-data breaches carry separate obligations, and sector regulators in finance and health have their own. The practical answer is to map your obligations once, in advance, and attach them to each system in your AI system inventory.
What should we log to make investigation possible?
Input, final output, model and prompt version, retrieved context identifiers, every tool call with arguments and results, guardrail decisions, latency and cost, plus a correlation id linking the whole run. Retain it long enough to cover your slowest detection path — ninety days is a sensible floor for most deployments.
What does an AI incident response plan cost to build?
Far less than most teams expect, because the expensive parts are the logging and evaluation capability you need anyway. Two weeks of focused effort produces a workable first version; the ongoing cost is one rehearsal a quarter and a monthly review of near misses.
Your First AI Incident Response Plan in Thirty Days
You do not need a mature programme to be dramatically better prepared than you are today. The first version should be short enough that people actually read it and specific enough that it changes what happens in the first hour.
Week one and two — inventory and severity
List every AI system in production with an owner, the decisions it can make, the systems it can write to, and whether it is customer-facing. Then write the severity matrix and get it agreed by someone who can authorise a shutdown. Those two artefacts are the spine of the AI incident response plan, and together they resolve most of the confusion in a real event.
Week three and four — kill switch and rehearsal
Document and physically test the stop procedure for your highest-risk system. Add one outcome monitor and one human report channel. Then run a ninety-minute tabletop, capture what broke, and fix the top three gaps. If you also hold an incident response retainer, make sure the provider knows the AI systems are in scope.
Then keep it alive
Review the plan whenever you deploy a new AI system, whenever a near miss clusters, and at least twice a year. An AI incident response plan is not a compliance artefact you produce once and file. It is the operating manual for the day something you deployed starts making confident, well-formatted, entirely wrong decisions on your behalf — and the teams that write it early are the ones still trusted afterwards. Setting that direction is part of a wider AI strategy conversation, not a side project for the engineering team alone.
References
NIST AI Risk Management Framework
NIST AI 100-1: Artificial Intelligence Risk Management Framework 1.0
NIST AI 600-1: Generative AI Profile
NIST SP 800-61r3: Incident Response Recommendations and Considerations
NIST AI 100-2: Adversarial Machine Learning Taxonomy
EU AI Act Article 73: Reporting of Serious Incidents
European Commission Regulatory Framework for AI
OWASP Top 10 for Large Language Model Applications
MITRE ATLAS: Adversarial Threat Landscape for AI Systems
NCSC Guidelines for Secure AI System Development
NCSC Incident Management Collection
ICO Guidance on Artificial Intelligence and Data Protection
OECD AI Incidents and Hazards Monitor
Google SRE Book: Managing Incidents
More AI coverage: explore Progressive Robot's AI Models, Tools & Releases hub — hands-on reviews, setup guides and benchmarks in one place.
