Understanding the Client's Use of AI
The Inventory Question
Begin with a question that sounds administrative and is actually diagnostic: does management know where models run? A client with a working answer can produce a list — which systems contain learned or statistical components, what each one decides or influences, who owns it, and what changed this year. A client without one will offer a tour of enthusiasm instead: the transformation programme, the vendor's roadmap, the pilot that finance is excited about. The difference is audit-relevant before any model is examined, because the inventory is itself evidence about the control environment. An organisation that cannot say where its models are cannot be monitoring them, cannot be managing their changes, and cannot have considered what happens when one of them is wrong. The inventory question costs one meeting, and its answer shapes the risk assessment more than any technical detail will.
- First question: does management know where models run, who owns each, and what changed this year
- A real answer is a list; a substitute answer is a tour of the transformation programme
- The inventory is evidence about the control environment before any model is examined
- An entity that cannot locate its models cannot be monitoring or change-managing them
The Red Flags
Four answers should raise the audit antennae. Nobody owns it: the model was a project, the project closed, and the outputs kept coming. No fallback: asked what happens when the model is wrong or unavailable, management describes the model again. Vendor-only understanding: the supplier is the sole party who can explain the behaviour, which converts a modelling question into a contractual one at the worst possible moment. And the deployment surprise: finance discovered the model after IT or operations had already put it into the flow of transactions, which tells you that decisions affecting the statements are being made outside the controllership's field of view. None of these is a misstatement, and none of them alone condemns the client. Each is a fact about how the entity governs things that touch the numbers — and that is precisely the material risk assessment is made of.
- No named owner: the project closed and the outputs kept coming, answerable to nobody
- No fallback: asked what happens when it is wrong, management describes the model again
- Vendor-only understanding turns a modelling question into a contractual one at the worst moment
- Finance discovering a deployed model late means statement-relevant decisions happen out of its view
What to Write Down
What goes in the file at this stage is unglamorous and valuable. The inventory as management provided it, or the note that management could not provide one — the absence is a finding about the environment, not a gap in your work. For each model that plausibly touches the statements: what it feeds, who owns it, what changed in the year, and what management does when it misbehaves. The red flags observed, recorded in neutral language and tied to the risks they inform, rather than left as corridor impressions. And the decisions taken: which model-touched areas the team treats as higher risk, where a specialist may be needed, and what the planned response is. Write it as you would any understanding-the-entity work. The point is that a reviewer can watch the reasoning travel from what was learned to what the plan does about it.
- File the inventory as given — or the note that none exists, which is itself a finding
- Per model: what it feeds, who owns it, what changed, and what management does when it misbehaves
- Red flags go in as neutral observations tied to the risks they inform
- A reviewer should watch the reasoning travel from what was learned to what the plan does
Prefer slides, quizzes, and saved progress? Read this lesson in the library — free, no sign-up.