Guide

How to assess your AI governance maturity

A method for measuring how mature your AI governance is. The seven areas to measure, the five levels each can sit at, how to score practice and what to fix.

How do you assess your organisation's AI governance maturity?

Measure the practices that make up AI governance, not the documents that describe them, across the seven areas that together decide whether an organisation can know what AI it runs, decide what is acceptable, control what actually happens and prove it to someone outside. Place each area on a five-level scale from Absent through Ad hoc, Defined and Managed to Optimised, using descriptions of what each level looks like in practice rather than a self-rating. Weight the three areas everything else depends on, inventory, enforcement and evidence, more heavily. Then treat the result as a map of where to work next rather than a grade, and act on the weakest heavily weighted area first.

To assess your AI governance maturity, place each of seven areas of practice on a five-level scale from Absent to Optimised, scoring how each practice actually runs rather than what is written down. Then act first on the weakest of the three areas everything else depends on, which are inventory, enforcement and evidence.

Most organisations have some AI governance. Few have all of it, and the parts they are missing are rarely the parts they expect. A maturity assessment is the structured way to find out, and the method is simple enough to run in an afternoon. This guide sets out the seven areas worth measuring, the five levels each can sit at, how to score honestly and what to do with the result.

The seven areas

AI governance is a set of practices, and the practices group into seven areas. Each answers one question about the organisation.

Accountability and oversight. Who owns AI outcomes, whether the board sees them, whether a defined body decides contested questions and whether those decisions are recorded in a way that survives the people who made them.

Policy and people. Whether the rules for AI use are written, usable and current, whether the people bound by them have actually been trained and whether a policy change reaches the people it affects.

Inventory and discovery. Whether the organisation knows what AI it uses, including tools staff have adopted on their own, AI embedded in purchased software and agents acting on its behalf, and whether the record is reconciled against what actually runs.

Risk assessment. Whether each AI system is assessed against a defined method, whether the assessment covers the people it affects as well as the risk to the organisation and whether AI risk sits inside normal risk management rather than beside it.

Controls and enforcement. Whether the controls that govern AI operate as mechanisms in the path of live traffic, or exist only as documents and good intentions.

Monitoring and incident response. Whether the organisation can see how its AI behaves in production, notice when something changes and handle an incident through to closure.

Evidence and assurance. Whether the organisation can show an outside party what it did and when, without reconstructing the story afterwards, and whether that evidence can be verified without taking the organisation’s word for it.

Three of these carry more weight than the others because everything else depends on them. You cannot govern what you have not found, so inventory comes first. Enforcement is where harm is prevented or not. Evidence is what an outside party asks for. An organisation with strong policy and weak inventory is governing a list, not its AI.

The five levels

Every practice sits somewhere on the same ladder. The labels are a guide. What matters is the description of what each level looks like for that specific practice, which is why a good assessment writes those descriptions out rather than asking for a number from one to five.

Absent. The practice does not exist. Nothing is written, nothing is done, nobody owns it.

Ad hoc. Something exists but it depends on individuals. It was done once, or is done inconsistently, and there is no owner or regular schedule. When the person who keeps the list moves on, the list stops.

Defined. The practice is documented, owned and followed. There is a written method, a named person and a repeatable process. This is the level at which an organisation could describe the practice to a regulator with a straight face.

Managed. The practice is operating, measured and maintained. It runs on a regular schedule, produces records and someone checks that it is working. The characteristic step from Defined to Managed is that the practice stops depending on people remembering.

Optimised. The practice is embedded in how the organisation works, largely mechanised, continuously reconciled against reality and improving from what it observes. Few organisations reach this across every area, and a result here is usually a strong programme with one or two areas still to arrive.

A sixth answer belongs on the ladder, which is don’t know. It scores as nothing, because for governance purposes a practice that nobody can point to is close to a practice that does not exist. It is a more useful answer than a guess, and where the don’t knows cluster is itself a finding.

How to run it

Decide the scope you can answer for. The whole organisation if you can speak for it, otherwise the division or programme you can. A narrow honest scope is worth more than a broad guessed one, and two narrow assessments from different divisions that disagree tell you something a single blended one would hide.

Score what operates, not what is written. This is the discipline that makes the exercise worth doing. A policy that exists but that nobody has read is Absent for the people bound by it. A control described in a risk register but not implemented anywhere is not enforcement. A board that has heard about AI once, after an incident, does not receive reporting. Answer for how the practice runs today.

Use don’t know as an answer. Guessing upward is the commonest failure of self-assessment. If you cannot say whether something exists, say so, and note who would know. Finding out is the first action on the list.

Run it separately with risk, legal and engineering. Have a colleague from each function complete the same assessment without conferring, then put the results side by side. Engineering usually scores enforcement and monitoring lower than risk does, because engineering knows what the gateway actually does. Risk usually scores policy higher than engineering does, because risk wrote the policy. The disagreements are the map of where the organisation’s picture of itself is inconsistent.

Act on the lowest weighted area first. Read the overall level, then ignore it. The useful part of the result is the area results and what the next level looks like in each. If inventory, enforcement or evidence is the weakest area, start there regardless of the overall number, because the other areas cannot be made real without them. In almost every organisation at the Absent or Ad hoc levels the first two actions are the same. Name someone accountable, and find out what AI is actually in use.

Reading the result

The overall level is a summary and behaves like one. An organisation with strong policy, a chartered committee and a documented risk method can land at Defined overall while nothing stands between its users and the models they call. The area results show that. Treat the overall as the headline and the areas as the work.

What the assessment cannot tell you is whether a practice you describe as operating actually operates, whether an assessment reached a sound conclusion or whether the people recorded as trained retained anything. It also cannot see the parts of the organisation you cannot see. It is a structured view of where governance is strong and where it is thin, and a set of descriptions of what the next level looks like. For proof that holds up outside the organisation, the companion guide on what makes AI evidence independently verifiable covers a different exercise.

How this runs in Aicura

Inventory is the area most results say to start with, and it is the part of Aicura that is free. The free AI register holds every AI system and agent with a named owner, and discovery reads the identity, device, cloud and code systems you already run to find the AI nobody mentioned. That page walks through signing up, connecting a source and building the register from what it finds.

A note on this page

This guide describes a method, and the method is neutral. It names no jurisdiction, regulator or standard, because maturity is about how practices operate rather than which obligation they serve. Which obligations apply to your organisation is a separate question, and the frameworks pages cover them by jurisdiction.

Questions

Is an AI governance maturity assessment the same as a compliance assessment?

No. A compliance assessment measures against a specific external obligation and produces a pass or a finding. A maturity assessment measures how well a practice operates regardless of which obligation it serves, and produces a level. An organisation can be highly mature and still non-compliant with a new rule it has not yet noticed, and it can be compliant with one rule while its governance is otherwise ad hoc.

What AI governance maturity level should we aim for?

There is no single target maturity level. The level that is right for an organisation depends on what its AI does, who it affects and what outside parties will ask of it. A firm whose only AI is a drafting assistant does not need Optimised evidence practices. A firm making automated decisions about individuals needs a great deal more than Defined. Use the levels to find the next step in each area, not to chase a number.

How often should we reassess our AI governance maturity?

Reassess AI governance maturity whenever something material changes, and otherwise about twice a year. Material changes include a new class of AI system, an agent deployed for the first time, a new obligation, an incident or a change in who is accountable. The value is in the movement between assessments, so keep the earlier results.

Can an AI governance self-assessment be trusted?

An AI governance self-assessment can be trusted only as far as the honesty of the person answering and the parts of the organisation they can see. It cannot tell whether a control described as operating actually operates. That is why the method scores practice rather than documents, includes don't know as an answer and asks different functions to answer separately. For proof that holds up outside the organisation, evidence a third party can verify is a different exercise.

Contact us about where you stand

If you would like a second view on your results, or help deciding which area to work on first, contact us.