The AI Learning Hub Journal

Where Pilots Die in Audit Firms

Most pilots die of governance before the technology can disappoint anyoneaudit firms have run enough pilots for the failure patterns to be recognisable — and none of them is technical THE GRAVEYARD — FOUR RECOGNISABLE GRAVESCLIENT DATA ACCESSNOBODY CLEAREDa pilot planned around realengagement files discovers,months in, that engagementterms, confidentiality duties andthe client’s own regulators standbetween the data and the toolMETHODOLOGY APPROVALAFTER BUSY SEASONcleared in April, when nobodywill risk learning it in the oneperiod the firm actually works —by the next busy season thelicence has lapsed and thechampion has moved onINSPECTION FEARthe partner’s asymmetry: savedhours against defending a novelapproach to an inspector with noprecedent — a firm that has notdecided how to explain reliancehas decided against relyingANOTHER ENGAGEMENT’SCHAOSa tool tuned on one engagement’sdocuments meets differentsystems, formats and scannedcontracts — the pilot’s resultswere real, and they wereresults about the pilotthe shared cause: the pilot was planned as a technology project, and in an audit firm the binding constraints were never technical WHAT THE SURVIVORS HAD — THREE UNGLAMOROUS FEATURES A NAMED OWNERone person answerable for thetool’s behaviour, its approvalstatus and its results —and currently in post A BOUNDED FIRST USEone procedure, one documenttype, one volunteer engagement— chosen so failure is cheap,contained and informative SIGN-OFF BEFORE ROLLOUTmethodology and qualityinvolved while the answer couldstill be no — not asked to blessa fait accompli afterwardsnone of this is technical, which is the point — adoption treated as an extension of how the firm already governs its work SURVIVORS EXTENDED THE FIRM’S EXISTING GOVERNANCE INSTEAD OF WORKING AROUND IT an unowned pilot dissolves the first time it produces a strange output and everyone assumes someone else is looking into it
Plan the governance first — data access, approval timing, the inspection story and the transfer claim are the actual project.

The Graveyard

Audit firms have now run enough AI pilots for the failure patterns to be recognisable, and most deaths happen before the technology gets a chance to disappoint anyone. The commonest grave is client data access nobody cleared: a pilot planned around real engagement files discovers, months in, that engagement terms, confidentiality duties and sometimes the client's own regulators stand between the data and the tool, and nobody had asked. Next to it lies methodology approval that arrives after the busy season: a tool cleared for use in April is a tool nobody will risk learning in the one period the firm actually works, so by the next busy season the licence has lapsed and the champion has moved on. Both deaths share a cause. The pilot was planned as a technology project, and in an audit firm the binding constraints were never technical in the first place.

  • Most pilots die before the technology gets a chance to disappoint anyone
  • Client data access nobody cleared: engagement terms and confidentiality duties discovered months in
  • Methodology approval landing after busy season means a tool nobody will risk learning until next year
  • The shared cause: pilots planned as technology projects when the binding constraints were never technical

The Quieter Deaths

Two quieter failures kill the pilots that survive the first year. The first is inspection fear. A partner weighing a new tool is weighing an asymmetry: the upside is saved hours, the downside is standing in front of an inspector defending a novel way of working with no precedent to point at. Rational partners retreat to the old way, and a firm that has not decided — at firm level — how it will explain tool reliance to its regulator has silently decided that its partners will not rely. The second is transfer failure: a tool tuned on one engagement's documents meets another engagement's chaos — different systems, different formats, a client whose contracts live in scanned images — and the accuracy that justified the rollout quietly does not come with it. The pilot's results were real. They were results about the pilot.

  • The partner's asymmetry: saved hours against defending a novel approach to an inspector without precedent
  • A firm that has not decided how to explain tool reliance has decided its partners will not rely
  • A tool tuned on one engagement's documents rarely survives another engagement's formats and chaos
  • Pilot results are results about the pilot — transfer to the next engagement is a separate claim

What the Survivors Had

The pilots that became practice share three unglamorous features. They had a named owner — one person answerable for the tool's behaviour, its approval status and its results, which sounds like bureaucracy until you watch an unowned pilot dissolve the first time it produces a strange output and everyone assumes someone else is looking into it. They had a bounded first use: one procedure, one document type, one engagement whose partner volunteered, chosen so that failure would be cheap, contained and informative rather than a firm-wide embarrassment. And they had methodology and quality sign-off before rollout, not after — the people who answer for the firm's system of quality management involved while the answer could still be no. None of this is technical, which is the point: the survivors treated adoption as an extension of how the firm already governs its work, not as an exception to it.

  • A named owner: one person answerable for the tool's behaviour, approval status and results
  • A bounded first use: one procedure, one document type, one volunteer engagement, cheap to fail
  • Methodology and quality sign-off before rollout, while the answer could still be no
  • Nothing on the list is technical — survivors extended the firm's existing governance rather than working around it

Prefer slides, quizzes, and saved progress? Read this lesson in the library — free, no sign-up.