Human oversight of AI: in the loop, on the loop, and oversight that works

Almost every AI framework asks for human oversight, and almost every organisation says it has it. The harder question is whether the oversight would actually catch anything. This guide covers the vocabulary, how to choose a level of oversight for each system, and how to design the reviewer’s job so the oversight is real.

The three positions

The terms come from control engineering and are now used loosely, so it is worth pinning them down.

• Human in the loop. The system proposes and a person decides. Nothing takes effect until someone approves it. A credit model that recommends a decline, which an officer accepts or rejects, is in the loop.
• Human on the loop. The system acts and a person supervises. They watch the system’s behaviour, receive alerts, and can pause or override it. A fraud model that blocks transactions automatically, with an analyst reviewing patterns and appeals, is on the loop.
• Human out of the loop. The system acts with no one reviewing individual decisions. People set it up and check it periodically. A spam filter is usually out of the loop.

Your AI register should record which position applies to each system, because so much else depends on it. Risk ratings, transparency duties and the controls you need all change with it.

What the frameworks ask for

The EU AI Act sets the most detailed expectations. Article 14 requires high-risk AI systems to be designed so that the people overseeing them can understand the system’s capacities and limitations, stay aware of the tendency to over-rely on its output, interpret the output correctly, decide not to use it or to override or reverse it, and intervene or stop the system. Article 26 then requires deployers, the organisations using those systems, to assign oversight to people with the necessary competence, training, authority and support.

Data protection law approaches the same question from the other side. Article 22 of the GDPR restricts decisions based solely on automated processing that produce legal or similarly significant effects. European data protection guidance treats human involvement as meaningful only where the reviewer has the authority and competence to change the decision and actually weighs the relevant information. The UK’s revised Articles 22A to 22D, in force since February 2026, keep safeguards including the right to obtain human intervention and to contest a significant decision.

Voluntary frameworks say the same thing in plainer terms. Australia’s Guidance for AI Adoption makes maintaining human control one of its six practices. The NIST AI Risk Management Framework asks organisations to define and document the human roles in AI oversight under its Govern and Map functions.

Step 1: Map the decision and its consequence

For each system, write down four things. What it decides or produces. Who is affected. How bad a wrong output is for them. How quickly a wrong output can be undone.

A marketing copy generator with a person editing every draft has low consequence and instant reversibility. A model that triages emergency calls has severe consequence and no reversibility. Most systems sit between, and the four answers place them.

Step 2: Choose the oversight level

Use the consequence and reversibility to pick a position.

• Severe and hard to reverse points to human in the loop, with a reviewer who can say no.
• Moderate, or high volume with recoverable errors, points to human on the loop, with monitoring, alerts and a fast way to intervene.
• Low and easily reversed can run out of the loop, with periodic sampling.

Record the reasoning. “On the loop, because individual declines can be appealed within 48 hours and volume rules out case-by-case review” is a defensible position. “On the loop” alone is not.

Watch for oversight drift. A system approved as in the loop often slides to on the loop as volumes grow and reviewers start bulk-approving. Treat that slide as a change that needs a fresh look.

Step 3: Design the reviewer’s job

This is where most oversight fails. A reviewer who sees only the system’s answer, has thirty seconds per case and is measured on throughput will agree with the system. That is automation bias, and Article 14 of the EU AI Act names it directly.

Design against it.

• Show the evidence, not just the answer. The reviewer needs the inputs the system used and, where possible, the reasons behind its output.
• Give them time. Set review targets that allow genuine consideration. If volume makes that impossible, the system is not really in the loop.
• Train them on failure. Show reviewers the kinds of errors the system makes, with real examples, so they know what to look for.
• Give them authority. The reviewer must be able to override without seeking permission and without penalty.
• Make disagreement cheap. If overriding takes five clicks and a justification while agreeing takes one, people will agree.

Some teams hide the system’s recommendation until the reviewer has formed a view, or seed known errors into the queue to check attention. Both are worth considering for consequential decisions.

Step 4: Build the intervention levers

Oversight on the loop depends on levers a person can pull. Pausing the system. Overriding a decision. Rolling back to a previous version. Tightening a limit. Stopping it altogether. For agents, suspending or re-scoping what they are allowed to do.

Write down who may pull each lever and how quickly. Then test them. A kill switch nobody has tried is an assumption, not a control. Decide in advance what the system does when a control cannot be evaluated, such as whether requests fail open and pass or fail closed and stop.

Step 5: Measure whether oversight is real

Oversight leaves traces you can measure. Track the override rate, the time reviewers spend per case, how often they escalate, and how often a later check finds an error the reviewer approved. Each has a warning sign.

• An override rate near zero for months suggests reviewers have stopped engaging.
• Review times of a few seconds on complex cases suggest rubber-stamping.
• No escalations suggests either a perfect system or a process nobody uses.

None of these proves a problem, but each is a reason to sample decisions and look. Report the measures alongside the system’s own performance, because oversight that degrades is a risk in its own right.

Oversight for agents

Agents push oversight from the output to the action. A person reviewing an agent’s email after it has been sent is not overseeing anything. For agents, oversight means bounding what the agent may do before it acts, deciding which actions always stop for approval, recording what it did, and keeping a working way to suspend it. The guide on governing AI agents covers this in more depth.

Where Aicura fits

The oversight position in Step 2 is recorded for every system in the Aicura register, and each risk assessment sets out the risk tier with the oversight that corresponds to it, so a change in position shows up as a new version of the system’s record. For systems on the loop, Pro adds the levers from Step 4. Guardrail rules can escalate a request to a person, each system behind a gateway carries a recorded choice of failing open or failing closed, and alerts flag a monitored figure crossing a limit you set and wait for someone to accept, dismiss or defer.

Guardian, Aicura’s automatic response feature, applies the same principle to the platform itself. For each condition it watches, you choose whether it only tells you, acts once a named approver agrees, acts and notifies, or acts, and every lever it can reach is one a person could already pull. How much autonomy a system gets stays a decision for your people.