Functions as Intended: The FRC's Review of How Audit Firms Certify Their Automated Tools, June 2025

In June 2025 the Financial Reporting Council, the UK's audit regulator, published a thematic review called Certification of Automated Tools and Techniques. Its subject is the technology audit teams use on an engagement "to perform risk assessment procedures and/or obtain audit evidence," which it calls automated tools and techniques, or ATTs. It reviewed the certification processes of "the six largest audit firms."

Its observations "do not constitute an expected set of processes and controls firms should have in place," and firms should consider how far they are relevant to their own circumstances. They also "reflect a snapshot of the processes and controls in place at the firms based on their responses to our IRL," its request for information, in the second quarter of 2024, and "These processes and controls may therefore have evolved." The report is meant for firms outside the review's scope too. Parts of it follow, not the whole.

What Certification Is For

ISQM (UK) 1 requires audit firms to establish quality objectives so that such tools are appropriately obtained or developed, implemented, maintained and used. The review's own definition of certification "broadly aligns to the key stages of a system development lifecycle," and runs from initial planning and needs analysis, through design and development and certifying the tool for implementation, to maintenance and monitoring. "The objective of the certification process is to verify the reliability of an ATT and its suitability for use in audits."

"It is crucial audit teams can rely on the outputs of ATTs to support their audit opinions." The use of these tools "has significant potential to improve audit quality," the regulator says, though that depends on the tools "producing consistently reliable outputs and being used routinely in the intended manner."

Certifying a Tool Centrally

The tools, the review says, "can broadly be divided into two categories." Some are developed for use across many audits, and their certification is "typically performed by a central team." Others are "bespoke, highly tailored or developed specifically for a particular audit," and their certification is "typically performed by the individual audit teams."

For the centrally certified tools, "All firms have processes in place for certifying ATTs prior to deployment," though their maturity varied and in some cases they "were not supported by formal documented policies." "All firms' certification processes included testing to ensure the ATT functions as intended and produces reliable outputs based on the input data." That may include comparing the expected output with what the tool produced, or reviewing the tool's logic or code.

The firms were consistent that engagement teams are responsible for assessing the completeness and accuracy of the data they enter into a tool, though in some cases the certification documents did not say so explicitly. "Only two firms explicitly outlined the limitations of the ATTs and/or restrictions of use within the certification templates." Other firms, the review says, would instead tell audit teams and users about such limits, where relevant, through separate training and guidance.

Some tools were in use before the formal processes existed. At some firms, certain of them had not been through the formal process; the firms explained that they would have been tested and assessed when they were deployed, though the supporting documentation may be less formal.

When the Tool Uses AI

Firms were beginning to deploy tools that use AI, the review says, including machine learning, "which can be applied to analyse full populations of journals or revenue transactions and identify unusual patterns or higher risk transactions." Tools using generative AI were, at the time of the review, limited to helping audit teams or aiding productivity, such as chat bots, rather than sitting within the tools the review covers.

"The use of AI within ATTs can present unique risks and challenges which need to be considered by the firms' certification processes." Those "may include, but are not limited to" the underlying algorithm or model, with its source, version number and the basis for its selection; the data used to train it; whether its outputs can be interpreted and explained; ethical risks, including bias; and further training and guidance for audit teams. The firms acknowledged that existing certification processes may not address those risks, and were planning changes. "Given such tools are now being used in audits, we encourage firms to implement the required changes to their certification processes as soon as possible."

Several of those questions are about the model itself: its version, its training data, whether its outputs can be explained. Where a firm did not build the model, the answers may have to come from whoever did.

The File, the Monitoring and Recertification

Central certification records are kept centrally. "While audit teams may refer to central certification procedures, there is no expectation for evidence of central certification procedures to be retained on audit files." For the bespoke tools, most firms required audit teams to complete templates and keep them on the audit file, though the review saw some examples of wording that "suggested that their completion was optional, rather than mandatory."

Monitoring "was generally restricted to a sub-set of more widely used ATTs, and primarily performed to understand how many audit teams were using the ATTs in their audits (typically for licensing purposes)." "There was no formal monitoring performed by the firms to quantify the audit quality impact of using ATTs." The review understands that this is an area the firms are looking to develop, and that "the use of ATTs may sometimes be captured in other activities such as root cause analysis." It encourages them to set policies or metrics for evaluating how the tools affect audit quality.

The identification of triggers for certifying a tool again "generally appeared to be informal, with firms typically relying on these being identified by ATT owners," who, the review understands, would usually be involved in approving changes to the tools. Some firms defined such triggers in their certification policies. And in general, the firms "do not monitor that audit teams are only using authorised and current ATTs," relying instead on preventive controls, such as restricting downloads and removing outdated versions.