AI is no longer just helping organisations automate work. It is increasingly shaping how people interpret information, compare options, assess risk and form judgements before decisions are made.
My recent article on AI-mediated judgement formation prompted a useful challenge. The response was valuable because it made a strong classical governance argument: governance principles have not changed. Organisations still need accountability, decision rights, evidence, independent challenge and assurance. AI is simply another input into an existing decision-making process. Therefore, the idea of a “judgement layer” is unnecessary terminology.
There is a lot in that argument I agree with.
Governance is not the act of forming judgement. Governance is the framework that defines accountability, authority, oversight, evidence, challenge and assurance around decisions. Those principles do not disappear because a new technology enters the organisation. Accountability still matters. Decision rights still matter. Assurance still matters. Evidence still matters. None of that has changed.
But that is not where the real disagreement sits.
The disagreement is about whether AI is merely another input into the existing decision-making process, or whether it is beginning to participate in the formation pathway of judgement itself. I think treating AI as just another input is the category error at the centre of much current AI governance thinking.
A dashboard gives us metrics. A report gives us analysis. A consultant gives us advice. An expert system applies bounded rules within a defined domain. All of these can influence judgement, and governance has always needed to consider evidence quality, assumptions, bias, independence and challenge.
But large language models are different in kind because they are not simply providing static information for later human evaluation. They are conversational, iterative, adaptive and increasingly embedded in the way people reason through ambiguity. They help frame the problem, suggest the premises, organise the evidence, generate the options, draft the recommendation and shape the language through which a decision is ultimately expressed.
That is not merely decision support.
It is upstream influence over judgement formation.
This is the structural shift I am trying to name. AI is moving inside the scaffolding of human reasoning. It is no longer sitting neatly outside the decision process, producing an output that a human then evaluates from a position of clean independence. In practice, people are asking AI tools what matters, what the risks are, how to interpret a message, how to structure an argument, how to respond to a board question, how to frame a control weakness, how to assess a vendor, how to summarise a legal obligation or how to think about a strategic trade-off.
By the time the formal decision is made, the judgement behind it may already have been substantially shaped.
This is why the phrase “human in the loop” is becoming less reassuring than it appears. It assumes a clean separation between machine output and human judgement: the system recommends, the human reviews, the human decides. But if the same system has helped the human understand the issue, narrow the options, draft the paper and develop the recommendation, then the human is not reviewing the output from outside the machine’s influence. The human is reviewing a decision pathway that may already have been mediated by the machine.
That does not mean human agency disappears. It does not mean accountability transfers to the model. In fact, the opposite is true. Because accountability remains human, we need much better visibility into how the accountable human’s judgement was formed when AI has materially influenced the pathway to that judgement.
This is the part traditional governance is at risk of missing.
Most AI governance still concentrates on the visible artefacts: the model, the data, the prompt, the output, the approval, the evidence pack, the final decision. Those things matter. But they do not fully capture the process layer in which AI shapes the interpretation, framing and reasoning that sit upstream of the final decision.
In other words, governance may be looking at the decision after the judgement has already been formed.
That is too late.
This has consequences well beyond cybersecurity or technology risk. Lawyers are using AI to test legal arguments and draft advice. Doctors are using AI to interpret symptoms and consider possible diagnoses. Executives are using AI to frame business risks, prepare board papers and explore strategic options. Risk, finance, procurement, HR, legal and cyber teams are beginning to use AI not only to produce documents, but to form positions.
The important question is no longer only whether the final decision was approved by the right person.
It is also how the judgement behind that decision was formed.
What did the AI frame? What did it filter? What evidence did it surface or exclude? What assumptions did it normalise? What alternatives were never considered because the machine made one pathway appear more coherent, more complete or more reasonable than the others? Was the human reviewer genuinely exercising independent judgement, or largely validating an AI-shaped pathway?
This is what I mean by the judgement layer.
It is not a replacement for governance. It is not a claim that governance principles have changed. It is the part of the decision process where information is interpreted, options are formed, trade-offs are evaluated and recommendations begin to take shape. AI is now entering that layer at scale.
That is why simply saying “governance has always required evidence, challenge and accountability” is true, but incomplete. The principles may be stable, but the location where they need to be applied has shifted.
Governance still needs to ask who owns the decision, who accepts the risk, what evidence supports it, who challenged it and how assurance was obtained. But in AI-mediated environments, it also needs to ask how the judgement was formed before the decision was made.
That is the real governance gap.
Not because AI creates a wholly new set of governance principles. It does not.
But because AI changes the control surface. It moves governance upstream from the final decision and visible evidence pack into the reasoning pathway that produced them.
If organisations continue to treat AI as merely another input, they will govern the artefacts they can see while missing the invisible process that increasingly shapes what people come to believe, recommend and approve.
Key Takeaway
Governance still asks who owns the decision and what evidence supports it. In AI-mediated environments it must also ask how the judgement was formed before the decision was made.
The principles of governance have not changed. But AI has changed where governance must look.