9 Signs Your AI Programme Is Creating Governance Risk Instead of Value
The programme reports green. The risk is accumulating anyway, in the gap between what has been deployed and what anyone can account for.
- AI governance risk accumulates quietly, in firms whose programmes are reporting green, because the questions that would expose it are not being asked internally.
- The nine signs below describe where the gap opens between what has been deployed and what anyone could account for if asked.
- The position worth holding is not “we have an AI policy”. It is “we know what is running, who owns each one, and what happens when it is wrong”.
Most mid-market AI governance risk is not the result of a bad decision. It accumulates because nobody was asked the question early enough to make a decision at all.
Tools get adopted departmentally. Models go into production without a formal review. Vendors add AI features to products the firm already uses. Colleagues find something useful and start relying on it. Each step is individually reasonable and none of it goes through anything resembling governance.
Meanwhile the external environment has moved. Regulators are asking more specific questions. Customers ask how decisions about them were made. Corporate buyers include AI questions in procurement. Insurers ask about it at renewal. Auditors have started looking.
The nine signs below describe where the exposure sits. Almost all of them are cheap to fix now and expensive to fix once someone else has found them.
Nobody can produce a list of what is actually running
Ask what AI the firm has in production and the answer takes days to assemble, arrives incomplete, and gets contradicted by someone in another department a week later.
That is not carelessness. AI arrives through many routes: built in-house, bought as a product, added by a vendor to something you already licensed, adopted by a team on a corporate card. No single route triggers a register entry, because no register exists.
The consequence is that every subsequent governance question is unanswerable. You cannot assess risk across things you cannot list. The fix is a register with an owner, a purpose, a risk rating and a review date for each entry. It is a fortnight of work and it is the foundation for everything else on this list.
Nobody is named as accountable for each one
Systems have technical owners. The data team owns the model. IT owns the platform. A vendor owns the product.
What is rarely named is the business person accountable for what it does — whether the outputs are appropriate, what happens when it is wrong, and whether it should still be running at all. That accountability usually sits nowhere, which becomes apparent the first time something goes wrong.
The fix is a named business owner per entry in the register. Not the person who built it or the one who administers it, but the one who is answerable for what it decides. It costs nothing and it changes how carefully things get reviewed.
Nobody has defined where a person must still decide
Some outputs go straight through. Others get reviewed. Which is which was usually determined by what was convenient to build rather than by what the consequences justify.
So the threshold is inconsistent. A low-consequence output gets checked because checking was easy to add, while something that reaches a customer directly goes through unreviewed because inserting a person would have slowed it down.
The fix is to decide deliberately, based on what happens if it is wrong: who sees it, what it costs, whether it can be undone, whether anyone would notice. Then write the threshold down. Most firms have never had that conversation, so the answer was set by whoever built the thing.
The risk is not that something goes wrong. It is that nobody could explain, afterwards, how it was supposed to work.
Colleagues are using AI tools nobody approved
People find tools that help them and start using them. Drafting, summarising, analysing, writing code. It is usually a sign of an engaged workforce rather than a reckless one.
The exposure is what goes into them. Customer data pasted into a consumer tool. Contract terms summarised by something with unclear retention. Personal information processed by a service the firm has no agreement with.
Banning it does not work — people revert to doing it invisibly. The fix is to provide an approved route that is genuinely good enough, be clear about what may and may not be put into anything, and check periodically what is actually being used. Most firms have a policy and no idea whether it is being followed.
Vendor AI features arrive without any review
Software you already licensed adds AI capability. It appears in an update. Sometimes it is on by default. Nobody assessed it, because nobody procured it — the decision was made by the vendor.
Those features process your data, produce outputs your people act on, and carry the vendor’s choices about training data, retention and where processing happens. You inherited all of it without a review.
The fix is to treat a vendor AI feature as a new deployment: assess it, record it, decide whether it stays on. That needs someone watching release notes, which sounds trivial and is the reason most firms do not do it.
Knowing what is running is cheaper than being asked and not knowing.
A free Executive AI Opportunity Assessment. Forty-five minutes, online, with your leadership team. We arrive with a model of how firms in your sector make money and where AI has moved those numbers — and you tell us where it is wrong.
Book a free assessmentNobody could explain a decision if a customer asked
A customer asks why they were declined, why they were priced that way, or why they were treated differently. Increasingly they have a right to a meaningful answer.
In most mid-market firms that answer would take weeks to construct, because nothing records which version of which model made the decision, on what inputs, under what rules. The information may exist in fragments across several systems and nobody has ever assembled it.
The fix is to keep a decision record for anything that affects a customer materially: what was decided, by which system, on what basis, when. It is a design decision at build time and an expensive retrofit afterwards.
Assembled on request, incomplete, and contradicted by another department within the week.
Technical owners exist. The person answerable for what it decides usually does not.
Adopted departmentally or on a card, outside anything that would have triggered a review.
Retrofitting governance onto something already in production, under time pressure, with someone watching.
This describes a pattern rather than a measured dataset. The useful test is your own: ask for a list of every AI system in production, with an owner against each, and see how long it takes to arrive and how confident anyone is that it is complete.
The regulatory position is being tracked by nobody in particular
The rules are moving. The EU AI Act is phasing in obligations by risk category. UK regulators are setting expectations sector by sector. Data protection authorities have published guidance. Sector regulators are asking specific questions.
In most mid-market firms nobody owns keeping track. Legal watches for legal changes. Compliance watches its own regulators. IT watches technology. AI sits across all three and belongs to none.
The fix is to name someone accountable for the AI regulatory position specifically, reporting periodically on what has changed and what it means here. And because these timetables have moved before, the useful discipline is to check the current position rather than rely on a date noted a year ago — including the dates in this paragraph.
Three things to do this month
- Ask for a complete list of AI in production, with a named business owner against each. However long it takes and however incomplete it is, that is your baseline.
- For the three with the most customer contact, ask what happens when the output is wrong. If the answer is vague, that is where to start.
- Check what AI features your existing vendors have switched on in the last year. Most firms find something nobody reviewed.
Nothing checks whether outputs are still good
Models were tested before deployment. They performed well. That was the evidence for going live.
What is rarely in place afterwards is anything checking whether they still perform. Inputs change. Customer behaviour shifts. A source system gets updated. The model carries on producing confident outputs based on patterns that no longer hold.
Degradation is gradual and invisible until something forces attention. By then the outputs have been unreliable for a while, and every decision made on them is questionable. The fix is monitoring with thresholds and an owner — the same discipline applied to any other production system.
Nobody owns AI governance as a job
The most structural of the nine. IT owns the platforms. Data owns the models. Legal owns contracts. Compliance owns the regulators. Risk owns the register. Each function does its part well.
What does not exist in most mid-market firms is anyone whose actual job is AI governance across all of it — the register, the ownership, the thresholds, the monitoring, the regulatory position, the vendor features.
That absence is what the other eight describe. Each is a gap between functions rather than inside one. The fix is not a new department. It is naming an existing executive as accountable, giving them a quarterly reporting line to the board, and enough authority to require answers from the functions. Most firms have an AI policy and nobody who owns it.
What this means
None of these nine came from a bad decision. They came from the absence of a decision — things arrived through routes that never triggered a review, and nothing was in place to catch them.
The pattern is consistent. Every function does its part competently. Nobody owns the space between them, which is exactly where AI sits. So the register does not exist, ownership is unassigned, thresholds are set by whoever built the thing, and the regulatory position is tracked by nobody in particular.
The economics strongly favour acting now. Building a register takes a fortnight. Assigning ownership takes a meeting. Deciding where a human must review takes an afternoon per system. Doing all of it after a regulator, an auditor, an insurer or a corporate customer has asked is a different exercise entirely, conducted under time pressure with someone watching.
Which is the same argument we make about value. Change the work before you change the technology — and decide how something should be governed before it is running, not after somebody asks.
Could you produce the list today, with an owner against each one?
Forty-five minutes with your leadership team, online, at no cost. We bring a working model of how firms in your sector make money and where AI has moved those numbers. You tell us where it is wrong. Then we tell you what we think — including “don’t invest”, if that is the honest answer.
Book a free assessment45 minutes · online · no cost · nothing to prepare
Continue reading
10 Data Foundations Missing Before AI Can Deliver
AI rarely fails on the model. It stalls on data nobody prepared.
For finance directors7 Governance Failures Quietly Undermining AI Returns
Why the return on AI spend is rarely defensible when someone checks.
For chief executives12 AI Readiness Gaps Boards Are Beginning to Question
The gaps boards now expect leadership teams to have an answer for.