The AI Learning Hub Journal

The Recurring Engagement Problem

Year two is not year one againa recurring engagement runs on rolled-forward understanding — and both sides of it can silently moveTHE CLIENT’S SIDE MOVEDyear onethe model as the file describes ityear twoa system the file no longer describesretrained, refitted on new data, or handed to a newowner — the control changed between your test dateslast year’s understanding describes a system that no longer existsYOUR OWN SIDE MOVED TOOyear onethe tool as the firm validated ityear twoa new model, the same interfacea new version on the vendor’s schedule — the screensthe team works in look exactly as they did last yearthis year’s flags are not produced by last year’s processTHE ROLL-FORWARD TRAP — “SAME TOOL AS LAST YEAR”true of the licencethe name on the invoice is stable≠false of the toolthe model behind the interface is nota version update moves what is flagged, what is missed and how confidently wrong output reads — with nothing visible on screenrolling reliance forward on the name rests this year’s evidence on last year’s validation of a different systemRELIANCE IS RE-EARNED ANNUALLY — WHAT TO RE-ASK, AND OF WHOMOF THE FIRMhas the tool changed —version, model,configuration — and wasit revalidated afterwards?OF THE CLIENTwas the model behind eachestimate retrained orre-owned — does last year’sunderstanding still hold?OF THE EVIDENCEdo this year’s flags meanwhat last year’s meant —same thresholds, samepopulations, same “clean”?OF THE CIRCUMSTANCESmoved jurisdiction, changeddata arrangements, alteredrules on what may enterwhich tool?then write the answers into the file — a reviewer can tell a file that asked from a file that assumed at a glanceCOMPARABILITY OF EVIDENCE IS A PROPERTY TO ESTABLISH, NOT ONE TO ASSUMEre-earning reliance costs one honest question — what changed — asked of the vendor, the firm’s tool owner and the client
Neither change announces itself — both arrive silently inside things that look unchanged, so they have to be found, not assumed away.

Year Two Is Not Year One Again

A recurring engagement runs on accumulated understanding: the team knows the client, the file rolls forward, and much of year two is testing whether year one's picture still holds. AI disturbs this from both directions at once. On the client's side, the model behind an estimate may have been retrained, refitted on new data or handed to a new owner, so the understanding documented last year describes a system that no longer exists. On the audit's side, the firm's own tool may have updated between years — a new version, a new model behind the same interface — so this year's flags are not produced by the process that produced last year's. Neither change reliably announces itself; both arrive silently inside things that look unchanged. The file must still support a reader comparing the two years, which means the changes have to be found, not assumed away.

  • Recurring audits run on rolled-forward understanding, and both sides of it can silently move
  • The client's model may have been retrained or re-owned since the file last described it
  • The firm's own tool may no longer be the process that produced last year's evidence
  • Year-on-year comparability of evidence is a property to establish, not one to assume

The Roll-Forward Trap

Same tool as last year is the recurring engagement's quietest trap, because the sentence can be true of the licence and false of the tool. The name on the invoice is stable; the model behind the interface is not. A version update can change what gets flagged, what gets missed and how confidently wrong output reads, without any visible difference in the screens the team uses. Rolling reliance forward on the strength of the name means resting this year's evidence on last year's validation of a different system. The same shape appears on the client's side of the file, where a control containing a model that updates is a control that changed between your test date and year-end — module three's territory. The general rule is short: reliance decisions are re-earned annually. Re-earning one costs a single honest question — what changed — asked of the vendor, the firm's owner for the tool, and the client.

  • Same tool as last year can be true of the licence and false of the tool behind it
  • A version change moves what is flagged and what is missed with no visible change on screen
  • Rolling reliance forward rests this year's evidence on last year's validation of a different system
  • Reliance is re-earned annually, starting with one question: what changed

What to Re-Ask Annually

The annual re-ask is short enough to be done and specific enough to matter. Of the firm: has the tool changed since last year — version, underlying model, configuration — and was it revalidated after the change? Of the client: has the model behind each significant estimate been retrained, refitted or re-owned, and does the understanding in last year's file still describe it? Of the evidence: do this year's flags mean what last year's meant — same thresholds, same populations, same definitions — or has a moved threshold quietly changed what clean means? And of the engagement's circumstances: has anything moved jurisdiction, changed data arrangements, or altered what may be put into which tool? Then write the answers into the file. A year-two file that shows the team asked these questions is a different document from one that assumed continuity, and a reviewer can tell them apart at a glance.

  • Ask the firm: did the tool change this year, and was it revalidated afterwards
  • Ask the client: was the model retrained or re-owned, and does last year's understanding still hold
  • Ask the evidence: do this year's flags mean what last year's meant, or did a threshold quietly move
  • Write the answers down — a file that asked is visibly different from a file that assumed

Prefer slides, quizzes, and saved progress? Read this lesson in the library — free, no sign-up.