The PRA's Model Risk Roundtable Slides Say the Current Periodicity of Ongoing Monitoring for AI and ML Models Is Not Frequent Enough

The Bank of England's Prudential Regulation Authority prepared a set of slides, dated October 2025, for a roundtable with firms on model risk management for artificial intelligence and machine learning. The slides say the authority "continues to treat model risk management (MRM) as a strategic supervisory focus area", and give the aim as sharing its current thinking on risks identified in AI and ML adoption, "in the context of implementing the expectations set out in SS1/23." SS1/23 is its supervisory statement setting out model risk management principles for banks. Parts of the deck follow, not the whole.

Risk Appetite and Model Tiering

The slides recall that a firm's board should set a model risk appetite that says what level and types of model risk it is willing to accept. AI and ML models, they add, introduce higher uncertainty than traditional models, "Due to their opaque nature and the lack of transparency".

On model tiering, the slide records an observation. "Instances were observed where firms' model tiering policies were not aligned with their model inventory submissions." For example, a policy may state that AI and ML models will have a minimum categorization of medium complexity, while in the corresponding inventory those models were classified as low complexity. The slide says inaccurate or absent information in model inventories leads to an "incomplete view of aggregate model risk, underdeveloped model risk appetite and a failure to adhere to the prescribed approach to risk management."

The same slide says that a model may have low materiality or complexity when assessed on its own, while deploying identical AI and ML techniques across multiple jurisdictions or portfolios "can result in a higher aggregate complexity or materiality." That, it says, reinforces the need for adequate challenge to the tiering approaches.

Nothing in that example fails loudly. A model sits in a lower tier than the firm's own policy asks, and the firm's view of its aggregate model risk is incomplete.

Explainability and Interpretability

Explainability and Interpretability (E/I), the deck says, "are key factors when determining a model's complexity under SS1/23." E/I techniques, the deck says, "are now commonly used to interpret AI and ML models by attributing the contribution of each input feature to the model's predictions." The slide then sets out "a critical limitation" that "our research highlights", citing research it lists as forthcoming, with publication expected in January 2026. The techniques assume that input features are independent, or at least uncorrelated, and the slide says that assumption "rarely holds in real-world big data". When features are correlated, the techniques may distort the ranking of which features matter most.

Its key message is that "Using E/I techniques on an AI / ML model is likely to mislead regarding importance of different features and therefore mislead regarding model behaviour and interpretation."

Data, Overfitting and Testing

On data, the slides say AI and ML models are particularly prone to overfitting, because of their high parameter count and sensitivity to noise, and may show significant performance deterioration on cases that differ from the data used to develop them. Standard validation splits may not detect the problem in a dataset that is not representative, because the training and testing sets come from the same sample, one that may not be representative of the target population.

On testing, the deck notes that the assumptions behind cross-validation and performance metrics, such as independence and stationarity, often do not hold for AI, ML or big data. And it says "models are frequently tested on clean, well-curated datasets, yet deployed in noisy, adversarial, or dynamic environments." The result can be testing outcomes that are overly optimistic.

Ongoing Model Monitoring, and the Questions for Firms

Firms often stated that the periodicity of their ongoing model performance monitoring is tier-dependent, the slides report, but broadly did not set quantitative expectations or limits in their policies. The risk named is that "AI and ML models, particularly models that can dynamically recalibrate, can quickly and cumulatively evolve beyond their validated risk appetite." The PRA's comment is that "The current periodicity of ongoing monitoring for AI and ML models is not frequent enough." For example, it says, six-month intervals may be insufficient to determine whether a dynamic model is performing as expected and staying within its risk appetite.

On performance degradation, the slides say AI and ML models may deteriorate so abruptly that little time is left for remediation, and that firms should consider pre-approved fallback models or challenger models "and, in extremis, kill switches." The slide adds that "Performance degradation is a consequence of poor model development testing."

The discussion slide asks: "What are your current challenges with AI and ML adoption, are existing MRM policies sufficient to mitigate the risks posed, and what further guidance is needed from us for AI and ML?"