When the Regime Changes
A Model Learns an Environment, Not a Law
Every model trained on market or credit data has learned a period: a level and shape of interest rates, a volatility regime, a default environment, a set of correlations, a liquidity condition, and whatever policy happened to be in force. None of that is a law of nature, and a model has no representation of the difference between a stable relationship and a coincidence that held for a decade. When the environment turns — a rate cycle, the end of a support scheme, a correlation inverting, a change in market structure — the model keeps producing outputs in exactly the same confident format. There is no field in the output that says the world it was fitted to has stopped existing, and no reason to expect one.
- Training data encodes a rate, volatility, default, correlation and liquidity environment, not a law
- Models cannot distinguish a durable relationship from a coincidence that held for a long time
- When the regime turns, format and confidence are unchanged and only accuracy moves
- The longer a relationship has held, the more it is trusted and the less anyone recalls what it rested on
Why the Failure Arrives at the Worst Moment
Regime changes are not random with respect to cost. The conditions that break a model — a volatility spike, a liquidity withdrawal, a correlation breakdown, a wave of defaults — are the same conditions that make being wrong expensive, and they arrive when positions, funding and management attention are already stretched. A risk model that understates the tail understates it precisely in the tail. A credit model calibrated on benign years is most wrong during the first bad one. This is the reverse of the ordinary software failure mode, where problems surface randomly and cheaply, and it is why model degradation cannot be left to be discovered by the loss it causes.
- The conditions that break models are the same conditions that make being wrong expensive
- Understating tail risk means understating it exactly when the tail arrives
- A model calibrated on benign years will be at its worst during the first bad one
- Discovery by loss is the default outcome unless monitoring is designed to fire before it
Monitoring That Fires Before the Loss
Module 2 covers what belongs in a model risk framework; here the point is timing. Input monitoring is the earliest signal, because what arrives changes before outputs are visibly wrong: feature distributions, missingness rates, new categories, application mix, and values outside the range the model was fitted on. Output monitoring comes next — score distributions, approval and flag rates, and how often humans override. Outcome monitoring is the most valuable and the slowest, because an outcome may take a year to observe and the exposure is booked long before then. So pre-commit thresholds: what movement escalates, who may suspend the model, and what the institution runs on meanwhile. Does Your AI Actually Work? covers the instrumentation; the finance-specific part is that lag.
- Input drift shows before output quality degrades — distributions, missingness, new categories, out-of-range values
- Output monitoring catches score, approval, flag and override shifts; outcome monitoring lags the exposure
- Pre-commit the escalation threshold and name who may suspend a model without reconvening a committee
- Decide the fallback before you need it: the manual or challenger process the institution reverts to
Prefer slides, quizzes, and saved progress? Read this lesson in the library — free, no sign-up.