The AI Learning Hub Journal

Change Control for Models That Update

Governing a model that keeps changing after go-livethe same gate for every change, whether it came from you or arrived from the supplierEvery proposed change enters one gate, and the first question is whether it alters the intended useSTEP 1ChangeproposedA retrain, a new build,a threshold tweak, a newpopulation or setting.STEP 2ImpactassessedDoes it alter intendeduse, risk, or who it issafe for? Answer first.STEP 3Revalidationplanned and runProportionate to thechange — local data andan agreed acceptance bar.STEP 4ApprovaldocumentedA named owner signs, withscope, limits and theevidence kept together.STEP 5Staged rolloutand monitoringA slice first, watchedagainst the bar, beforeit becomes the default.ROLLBACK AVAILABLE AT EVERY STAGE — AND REHEARSED, NOT ASSUMEDMonitoring after rollout feeds the next proposed change straight back into the same gateA LOCKED MODELBehaviour only changes when somebody changes itEach version is a discrete, testable artefactRevalidation is an event you can plan and scheduleThe burden sits on keeping versions traceableRisk: it silently ages while the setting moves onA CONTINUOUSLY LEARNING MODELBehaviour can change without any release happeningThere may be no frozen version to test againstMonitoring has to be continuous, not periodicChange control must cover the update rule itselfRisk: drift and improvement look identical at firstEducational orientation only — the actual approval route depends on the setting and the product
A model that can change needs a gate that never does — the governance burden follows the update mechanism

The Frozen Model and Its Alternative

Traditional device regulation assumes a product that does not change: it is assessed, approved, and then behaves the same way until the manufacturer submits a modification. Machine learning models sit awkwardly with this. They can be retrained on new data, adapted to local populations, or updated continuously — and each of those is precisely the behaviour the frozen-model assumption excludes. Most clinically deployed models are therefore locked: they do not learn in the field, and updates arrive as discrete versions. That is a deliberate safety choice, not a technical limitation, because a model that changes between one patient and the next cannot be meaningfully validated, audited, or reasoned about after an incident.

  • Device frameworks assume a product that behaves consistently after approval
  • Most clinical models are locked and updated as discrete versions rather than learning in the field
  • Locking is a safety and auditability choice, not a technical shortcoming
  • A continuously changing model cannot be meaningfully validated or investigated after harm

Planning Change in Advance

Regulators have developed the concept of specifying anticipated modifications up front: the manufacturer describes in advance what kinds of change it expects to make, the methods and data it will use, and the performance criteria the modified model must meet, so that pre-agreed changes can be made without a fresh submission each time. The logic is to make change predictable and bounded rather than pretending it will not happen. The practical significance for an institution is that a product may legitimately change after you deploy it, within an envelope you should understand. Ask what may change, what triggers a change, how you will be notified, and what evidence accompanies each version.

  • Anticipated changes, methods, and acceptance criteria can be specified in advance and agreed
  • The aim is bounded, predictable change rather than a pretence of stasis
  • Ask what can change, what triggers it, how you are notified, and what evidence comes with each version
  • In the US this mechanism is the Predetermined Change Control Plan, which the FDA has addressed in dedicated guidance

What the Institution Must Track

Change control does not end at the manufacturer. The institution needs to know which version is running, when it changed, what changed, and whether the local validation still holds — and it needs to know this without depending on a vendor release note nobody reads. Silent updates are a genuine operational risk: a tool can behave differently on Monday than it did on Friday with no local notification, and the first sign may be a change in clinician override rates. Version identity should be recorded alongside outputs so that any retrospective investigation can determine which model produced a given result. Re-validation triggers should be defined in advance, along with who is responsible for executing them.

  • Record model version alongside outputs so past results can be attributed to a specific model
  • Silent vendor updates can change behaviour with no local notification
  • Define in advance which changes trigger re-validation, and who performs it
  • Contract for advance notice of material model changes rather than relying on release notes

Prefer slides, quizzes, and saved progress? Read this lesson in the library — free, no sign-up.