The AI Learning Hub Journal

Controls When Software Decides

A model inside a control breaks “automated means consistent”test-of-one was always a test of one plus an assumption of constancy — and the assumption is the part learned logic removesTHE SAME CONTROL IS NOT THE SAME CONTROL ALL YEARyour test date — the filter as tuned in Marchyear-end — a different control in substancewritten logic — the same all yearlearned logic — retrained, recalibrated, adapting to volumetime →a matching threshold that adapts to volume is a different control in the busy season than it was in the quiet oneCHANGE MANAGEMENT BECOMES THE CONTROL THAT MATTERSwho may change orretrain the model,and on what authoritywhat approval achange requires,before it shipschanges logged withdates, reasons andnamed approversbehaviour testedbefore and aftereach changewhether an emergencypath bypasses allof the abovethe bypass, if it exists, is where the year’s real changes probably wenta change log with named approvers gives the team something to test — enthusiasm about agility does not“THE VENDOR PUSHES UPDATES AUTOMATICALLY”the relied-upon control is operated by a thirdparty, under a contract nobody in the room hasread recently, on a schedule the client neitherapproves nor logsthe control sits outside the client’s own approval and loggingDRIFT — A CHANGE WITH NO CHANGE TICKET AT ALLthe world moves, the data mix shifts, and thethresholds quietly start misfiring — whetheranyone would notice is monitoring, a controlsquestion of the most classical kindask for the last breach — investigation and recalibration means it is aliveLOGIC THAT UPDATES IS A CONTROL THAT CHANGES BETWEEN THE TEST DATE AND YEAR-ENDa monitoring report nobody reads is a control that exists in the design, not in the year under audit
What changed, when, and under whose authority is now the centre of the controls work, not its afterthought.

Automated No Longer Means Consistent

Controls testing has long rested on a comfortable asymmetry: manual controls vary, so you test many instances; automated controls repeat, so you test one and lean on it, provided change management holds. A model inside a control quietly breaks that asymmetry. Logic that is retrained, recalibrated or continuously learning is not the same logic all year. A fraud filter tuned in March behaves differently in November; a matching threshold that adapts to volume is a different control in the busy season than it was in the quiet one. The test-of-one was never truly a test of one — it was a test of one plus an assumption of constancy, and the assumption is the part that learned components remove. None of this makes such controls unauditable. It makes the question of what changed, when, and under whose authority the centre of the controls work rather than its afterthought.

  • Test-of-one was always a test of one plus an assumption of constancy
  • Retrained or adaptive logic is not the same control between your test date and year-end
  • A filter tuned in March and the same filter in November are different controls in substance
  • The controls question becomes what changed, when, and under whose authority

Change Management Becomes the Control

Once the logic can move, the discipline around movement is what stands between the test date and year-end. The questions are recognisable from ordinary IT change territory, tightened for models: who may change or retrain the model, what approval that requires, whether changes are logged with dates and reasons, whether behaviour is tested before and after, and whether an emergency path exists that bypasses all of the above — because the bypass, if it exists, is where the year's real changes probably went. A client that can produce a change log with named approvers has given the team something to test. A client that says the vendor pushes updates automatically has said something different: the control the team hoped to rely on is operated by a third party, under a contract nobody in the room has read recently, on a schedule the client neither approves nor logs.

  • Who may retrain, what approval it takes, what is logged, what is tested before and after
  • The emergency bypass, if it exists, is where the year's real changes probably went
  • A change log with named approvers is testable; enthusiasm about agility is not
  • Vendor-pushed updates mean the relied-upon control is operated outside the client's own approval

Drift Is a Controls Question

A model can change with no change ticket at all: the world moves, the data mix shifts, and yesterday's thresholds quietly start misfiring — drift, in the trade's vocabulary. The instinct is to file this under modelling, somebody else's speciality. Resist it. Whether anyone would notice the model going wrong is a control question of the most classical kind — it is monitoring — and its absence means errors accumulate until something downstream hurts enough to investigate. So the auditor asks controls questions. Does management compare model output against outcomes, at what frequency, against what tolerance, and what happened the last time a threshold was breached? A breach that led to investigation and recalibration is comforting evidence that the monitoring is alive. A monitoring report that nobody reads is a control that exists in the design and not in the year under audit.

  • Drift changes a model with no change ticket — the world moves and the thresholds misfire
  • Whether anyone would notice is monitoring, a controls question in its most classical form
  • Ask for the last breach: investigation and recalibration is evidence the monitoring is alive
  • A monitoring report nobody reads exists in the design, not in the year under audit

Prefer slides, quizzes, and saved progress? Read this lesson in the library — free, no sign-up.