Article 14 of the EU AI Act requires that high-risk AI systems be designed so that humans can effectively oversee them. The obligation sits with deployers: assign a capable person, ensure they understand the system’s capabilities and limitations, make certain they can intervene when necessary. On paper, the oversight loop is closed. In practice, it contains a structural gap that the Act does not address and that is wider precisely where it matters most.
What the Act Actually Requires
Article 14(4)(b) names automation bias explicitly — the tendency to rely on or over-rely on AI outputs without adequate independent judgment. This is unusual in legislative drafting; it signals that the drafters understood the risk. What they required in response was awareness: the human overseeing the system must be aware of the possibility of automation bias. What they did not require was any mechanism to verify that awareness is functional, or that the underlying professional competence the human brings to their oversight role is preserved over time.
This produces a distinction that the Act’s architecture treats as one problem but is actually two.
Acute Versus Chronic Competency
Automation bias is an acute phenomenon. The human is present, capable in principle, but defers to the machine’s output in the moment without adequately interrogating it. This is a cognitive failure that oversight design, training, and awareness requirements can plausibly address — the tools Article 14 provides are at least pointed at the right problem.
Skill atrophy is a chronic phenomenon. A human who stops exercising independent judgment over time may lose the capacity to exercise it reliably. By the time the AI system produces an output that genuinely requires an informed override, the overseer’s ability to evaluate whether the override is correct may have degraded in ways that are not visible to the deployer, the regulator, or the overseer themselves.
Article 14 has tools — imperfect ones — for the first problem. It has essentially nothing for the second. The recital language on awareness, the training requirements linked to Article 4, the AI literacy obligation — these address the cognitive disposition of the person at a point in time. None of them address whether that competency is preserved as AI systems progressively substitute for the judgment they are nominally supporting.
Assuming chronic competency is not merely legally acceptable under the current Act. It is structurally required by it, because the Act offers no alternative mechanism for verifying that the human in the oversight role is still capable of performing it six months or three years after deployment.
Where Independent Professional Frameworks Step In
The practical severity of this gap varies significantly depending on whether the deployer operates in a domain with an independent professional competency infrastructure.
Medicine
In medicine, licensing bodies, revalidation requirements, clinical audit, peer review, and governance structures create a secondary accountability layer. A clinician whose independent judgment degrades over time is not necessarily uncatchable — audit trails, morbidity and mortality conferences, and peer comparison provide mechanisms that exist entirely outside the AI Act. The Act’s oversight gap in clinical settings is real, but it is partially mitigated by regulatory architecture that predates AI.
Law
In law, bar association membership, professional indemnity obligations, and the duty of care to clients create financial and disciplinary incentives to maintain independent competency. A legal error enabled by uncritical reliance on AI output is still the lawyer’s liability. Malpractice exposure disciplines oversight behaviour in ways that Article 14 alone does not.
Journalism
In journalism, the framework is weaker institutionally but strong culturally. Editorial verification workflows, the four-eyes principle, legal review exposure, and corrections accountability create a professional environment in which checking and questioning outputs is embedded practice — not because regulators require it, but because errors carry reputational and sometimes legal consequences. The risk AI introduces in journalism is not so much competency atrophy as instinct erosion: verification feels redundant when the machine has already done it. Errors happen even under careful editorial practice. The concern with AI assistance is that it removes the friction that makes verification feel necessary.
The SME Gap
None of this applies in the same way to the domains where many SMEs actually deploy high-risk AI systems.
Take hiring decisions as one illustration. HR managers using AI for job screening are not professionally licensed or subject to routine revalidation in the way doctors or lawyers are, and there is no profession-specific body mandating periodic demonstration of competency. Their use of AI is still constrained by employment, discrimination, and data-protection law — in the EU through the AI Act itself alongside GDPR obligations, in the US through EEOC enforcement of existing anti-discrimination statutes, in the UK through employment tribunals with authority to scrutinise whether AI-informed decisions met fairness standards. External review can and does occur: through regulatory audit, required risk assessments, documentation duties, and compliance reviews, particularly for high-risk recruitment tools.
What is absent is not accountability in principle, but a profession-specific competency infrastructure — the kind that, in medicine or law, creates ongoing pressure to maintain the independent judgment that oversight requires. The external checks that apply here are triggered by outcomes: a discrimination claim, a regulatory investigation, a failed audit. They are retrospective and adversarial by design. That is a structural feature of how accountability works in this domain, not an absence of accountability altogether — but it is a meaningful difference from the continuous, profession-embedded pressure to maintain competency that exists in credentialed fields.
The same pattern holds across a range of SME deployment contexts — customer credit scoring, access decisions, automated performance evaluation. The accountability mechanisms exist, but they are outcome-triggered rather than competency-maintaining. No external framework routinely asks whether the human assigned to oversight is still capable of performing it.
The Counterintuitive Inversion
This produces a result that runs against the intuitive reading of the risk landscape. The instinct is that high-stakes domains — medicine, law, regulated finance — are where the Act’s oversight gaps are most dangerous. But those are also the domains with the most independent regulatory infrastructure to catch what the Act misses.
The Article 14 gap is actually widest in the lower-credential contexts where many SME obligations land, where the stakes per individual decision appear smaller but aggregate across large volumes, and where no external framework exists to catch competency degradation that the Act does not require anyone to measure.
What SMEs Should Do
The Act does not require competency verification over time. That does not mean the risk is not real, or that the operational and legal exposure from getting this wrong is manageable.
- Treat oversight assignment as a role description, not just a job title. Document what competency the assigned person is expected to bring — not the title, but the specific capability to evaluate the AI system’s outputs independently of those outputs.
- Build in deliberate friction. If the AI system produces a recommendation, the oversight process should require the assigned person to form an independent assessment before seeing the AI output, not after. This is organisationally inconvenient. It is the only reliable method for maintaining the judgment the oversight role is supposed to supply.
- Recognise that oversight competency can atrophy without anyone noticing. Build periodic review into your oversight arrangements — not just logging that a human was present for decisions, but examining whether the human’s independent assessments diverge from the AI’s outputs at a reasonable rate. A human overseer who agrees with the AI system in ninety-eight percent of cases may be performing excellent oversight. Or they may not be performing oversight at all. The difference matters and it is not detectable from the log alone.
- When procuring AI systems, ask vendors specifically what mechanisms exist to support genuine independent evaluation. Stop buttons and override procedures satisfy the Act’s design requirements. They do not address whether the person reaching for the stop button can correctly identify the moment at which reaching for it is warranted.
The EU AI Act’s human oversight framework is a serious attempt to address a genuine risk. Article 14 is better drafted than its critics sometimes allow — the explicit attention to automation bias, the insistence on real competency rather than job title assignment, the link to Article 4 literacy obligations.
But the framework was built to ensure that human oversight is present. It was not built to ensure that the human at the centre of that oversight remains capable of performing it. That gap is the deployer’s problem to manage, with or without regulatory guidance telling them to.
