How to respond to an AI incident

What counts as an AI incident

An AI incident is an event where the development, use or malfunction of an AI system leads to harm. The OECD groups the harms that make an event an incident into injury to people’s health or safety, disruption to critical infrastructure, breaches of human rights or of laws that protect people’s rights, and damage to property, communities or the environment. An event that has not yet caused one of those harms but could plausibly lead to it is an AI hazard rather than an incident.

The distinction sets the threshold for when your response process runs. A model that starts drifting is a hazard. The same model approving loans it should have declined, or rejecting a group of applicants it should not have, is an incident.

An AI incident is not only an outage. A system that keeps running while producing wrong, unfair or unsafe results is often the more serious case, because it can go unnoticed for longer. The events that usually meet the threshold include a system making decisions that harm people or breach their rights, a model behaving in a way its owners did not intend or cannot explain, a system exposing or misusing personal information and a tool being used outside the scope it was approved for.

Before an incident: be ready for one

A mature response depends on decisions you make before anything goes wrong. Agree who the accountable owner for AI incidents is, and who they can call on. Decide what severity levels you use and what each one commits you to. Write down, for each jurisdiction you operate in, the point at which an incident becomes reportable to a regulator or to an affected person, so that call is not being made under pressure for the first time. Keep this alongside your AI register, so that when a specific system is involved you already know who owns it and what it does.

Triage the event

When something is reported, the first task is to decide whether it is an incident, a hazard or noise, and how serious it is. Record what was seen, which system is involved, when it started and who raised it. Assign a severity using the levels you set in advance. Triage is also the point where you start the record, because the earliest facts are the ones most likely to be lost.

Contain the harm

Containment is about stopping the incident getting worse while you work out what caused it. Depending on the system, that can mean pausing it, routing decisions to a human, rolling back to a known-good version or narrowing what the system is allowed to do. Note what you changed and when, because a containment action taken quickly is easy to forget and important to be able to explain later.

Investigate the cause

With the harm contained, work out what actually happened. Look at the decision the system made, the input data behind it, the model version in use and any recent change to the system, its data or the vendor’s model. The aim is a cause you can state plainly and evidence, not a guess. Keep the working, including the versions and data you examined, so the conclusion can be stood behind rather than merely asserted.

Remediate

Remediation is the fix, both the immediate one and the durable one. The immediate fix restores safe operation. The durable fix addresses the cause so the same incident does not recur, which might be a change to the model, the data, the controls around the system or the approval it was running under. Record what you changed and the reasoning, so the link between the incident and the fix is visible.

Notify where you are required to

Notification duties depend on where you operate, what sector you are in and what the incident involved. They are time-bound, so work out early which ones apply, and record the assessment you made and the basis for it whether or not you end up notifying.

Everywhere. Check your contracts first. Customer and supplier agreements often carry their own notice periods for security or service incidents. Sector regulators in financial services, health and critical infrastructure commonly have incident reporting rules of their own, and where personal information is involved, data protection law usually applies.

European Union and United Kingdom. Where an incident involves a personal data breach, the GDPR and the UK GDPR require the controller to notify the supervisory authority without undue delay and, where feasible, within 72 hours of becoming aware of it. Where the breach is likely to result in a high risk to people, they must also be told without undue delay. The EU AI Act adds serious incident reporting for high-risk AI systems under Article 73. A provider must report a serious incident to the market surveillance authority no later than 15 days after becoming aware of it, with shorter limits for a death and for widespread or critical infrastructure incidents. A deployer, the organisation using the system, that identifies a serious incident must inform the provider immediately. These duties apply from when the high-risk obligations apply, which the 2026 Digital Omnibus amendments set at 2 December 2027 for stand-alone high-risk systems and 2 August 2028 for AI built into regulated products.

Australia. Where an incident involves personal information and amounts to an eligible data breach, the Notifiable Data Breaches scheme requires you to notify the OAIC and the affected individuals as soon as practicable, after assessing a suspected breach within 30 days. APRA-regulated entities have further clocks. CPS 234 requires APRA to be notified within 72 hours of becoming aware of a material information security incident. CPS 230 requires notice within 72 hours of an operational risk incident likely to have a material impact, and within 24 hours of a disruption to a critical operation outside tolerance. AI incidents and the NDB scheme covers when an AI incident is also a data breach.

Review and close

After the incident is contained and fixed, review it. Confirm the cause, confirm the fix held and capture what the incident revealed about the system and about the process. This is where near misses earn their place, because the lesson from an event caught in time is as useful as the lesson from one that was not. Close the incident with a record that a person who was not in the room could follow.

Keep a record that holds up later

The record a response leaves behind is what lets you show, afterwards, that you handled the incident properly. For an AI incident, the record that tends to matter covers the decision or behaviour that triggered the incident, the system involved and its version, the data behind the decision, who was affected and how, the containment and remediation actions and when they were taken, the basis on which you assessed each notification duty and the review that closed the incident. Capture these as the response runs. A record assembled weeks later from memory is both weaker and harder to stand behind.

The test is simple. A competent person who was not involved should be able to read the record and understand what happened, what you did and why, without needing to ask you.

Keeping the record matters as much as capturing it. ISO/IEC 42001 expects the activities of an AI management system to be documented and records to be kept and controlled, because a record that could have been changed after the event carries less weight when it counts. A reviewer’s first question is whether the record they are reading is the record as it stood at the time. Answering that well is what separates a record that reassures from one that raises further questions.

Where Aicura fits

In Aicura an incident usually first shows as an alert, a monitored figure crossing a limit you set on the system involved. Guardian, Aicura’s automatic response feature, can be set up to open an incident or switch the system to a stricter set of guardrail rules when a condition you define holds, with as much human approval as you choose. A gateway can also be set to fail closed for that system while you investigate, refusing its AI traffic rather than letting it through unchecked. The incident itself is raised against the AI systems it affects, so the response stays linked to each system’s owners, versions and assessments, and a workflow carries it through assessment, notification drafting where warranted and a final report.

Aicura helps you run and record the response. It does not decide the incident’s severity, whether you must notify or whether you handled it well. Those calls stay with you and your accountable owner.

Sources

• AI Risk Management Framework (AI RMF 1.0), NIST, January 2023, in particular MANAGE 4.3 on tracking, responding to, recovering from and documenting incidents
• Defining AI incidents and related terms, OECD, 2024
• ISO/IEC 42001:2023, Artificial intelligence, Management system
• Regulation (EU) 2016/679 (GDPR), Articles 33 and 34
• Regulation (EU) 2024/1689 (EU AI Act), Articles 26 and 73, as amended by the 2026 Digital Omnibus
• Notifiable Data Breaches scheme, Part IIIC of the Privacy Act 1988 (Cth), OAIC
• APRA Prudential Standards CPS 234 Information Security and CPS 230 Operational Risk Management

A note on this page

This is general information, not legal advice. Notification duties turn on the facts of each incident and the laws, regulators and contracts that apply to your organisation. Read the primary sources, and take your own advice on how they apply to you.