Accountability and the Governance Committee

Who Is Accountable When AI Contributes to Harm

There is no single answer, and anyone offering one confidently is oversimplifying. Responsibility is distributed across at least four parties: the manufacturer, for the product performing as specified for its intended use; the institution, for selecting it, validating it locally, integrating it safely, training staff, and monitoring it; the clinician, whose professional judgement and duty of care are not transferred by using a tool; and, in some framings, the regulator that authorised it. Legal treatment varies by jurisdiction and is still developing. What is stable is the professional position: a clinician who acts on an AI output remains responsible for that decision, and "the system said so" has never been an adequate account.

• Responsibility is distributed: manufacturer, institution, clinician, and regulatory oversight
• Using a tool does not transfer professional duty of care
• Legal treatment varies by jurisdiction and remains in development
• Institutions carry selection, validation, integration, training, and monitoring responsibility

What a Governance Committee Actually Does

An effective AI governance function is operational rather than advisory. It maintains an inventory of every AI-containing system in the institution, including tools procured as ordinary software. It runs an intake process so new proposals are assessed before purchase rather than after. It sets the local validation standard and reviews the results. It approves specific intended uses and records what is permitted. It defines monitoring requirements and receives the reports. It handles incidents. And it retires tools. A committee that only reviews proposals and never monitors, retires, or suspends anything is a procurement gate wearing a governance label, and it will not catch the problems that matter.

• Maintain an inventory — including AI arriving inside systems bought as ordinary software
• Assess at intake, before purchase commits the institution
• Set the local validation standard, review results, and record approved intended use
• Receive monitoring reports, handle incidents, and actually retire tools

Composition and Authority

Two failure modes recur. A committee that is purely clinical cannot assess data protection, procurement, or technical questions. A committee that is purely technical approves things clinicians will not use and misses clinical risk entirely. Useful composition spans clinical specialties that will use the tools, informatics, data protection and information governance, risk and quality, procurement, and — increasingly expected — patient representation. Authority matters more than composition: the committee must be able to say no, and to suspend a live deployment without renegotiating its mandate. If a tool can be adopted by going around it, the committee documents decisions rather than governing them.

• Span clinical, informatics, data protection, risk and quality, procurement, and patient representation
• The committee must be able to refuse and to suspend a live deployment
• If tools can be adopted around the committee, it documents rather than governs
• Give the standing monitoring function resource — review meetings alone do not detect drift

The Lifecycle View

The most practical way to organise all of this is as a lifecycle every tool passes through, with named owners at each stage: intake and triage, where the proposal is characterised and risk-classified; evidence review, covering regulatory status, published evidence, and its limits; local validation against pre-committed criteria; approval of a specific documented intended use, with defined permitted and prohibited uses; controlled deployment with training and clear escalation routes; ongoing monitoring with defined metrics, reporting cadence, and suspension triggers; and eventual retirement, which needs planning because clinical workflows come to depend on tools quietly. Most institutional failures are not decisions to do the wrong thing — they are stages nobody owned.

• Intake and triage, evidence review, local validation, approval, deployment, monitoring, retirement
• Name an owner for each stage — unowned stages are where failures accumulate
• Record permitted and prohibited uses, not just approval
• Plan retirement: workflows silently come to depend on tools that were never meant to be permanent

Try It Yourself

Governance becomes real when the questions are written down before anyone is committed to an answer. Draft them for a pilot your institution could plausibly run.

Proposed tool and intended use:
Intake — what is the risk classification, and who assigns it?
Evidence — what exists, and what are its limits?
Local validation — what sample, what comparator, what stop rule?
Approval — what use is permitted, and what is prohibited?
Deployment — who is trained, and what is the escalation route?
Monitoring — which metrics, what cadence, whose desk?
Retirement — what triggers it, and who decides?