The six largest UK accounting firms do not formally monitor how automated tools and artificial intelligence impact the quality of their audits, the regulator has found, even as the technology becomes embedded across the sector.

The Financial Reporting Council on Thursday published its first AI guide alongside a review of the way firms were using automated tools and technology, which found “no formal monitoring performed by the firms to quantify the audit quality impact of using” them.

The watchdog found that audit teams in the Big Four firms — Deloitte, EY, KPMG and PwC — as well as BDO and Forvis Mazars were increasingly using this technology to perform risk assessments and obtain evidence.

But it said that the firms primarily monitored the tools to understand how many teams were using them for audits, “typically for licensing purposes”, rather than to assess their impact on audit quality.

The tools included those using artificial intelligence, such as machine learning. The regulator said that some firms were also deploying generative AI technologies, such as chat bots, although these fell outside the review’s scope.

All the firms bar one also did not have key performance indicators for the tools they used, the FRC found. The work was prompted by the regulator’s audit quality review team, which had flagged an increase in the use of the technology.

AI is rapidly transforming the audit sector, with firms including the Big Four heavily investing in AI-powered tools to enhance efficiency across multiple stages of the audit process. But the FRC said AI could also present “risks and challenges” in audits including ethical issues and the potential for bias in tools’ outputs.

Its scrutiny over the use of AI technology comes after the FRC criticised BDO and Forvis Mazars last year for shortcomings in their audits for the fourth straight year, threatening “stronger action”.

Meanwhile EY said it would invest about $2bn from 2021 to improve the quality of its audits following scandals including the collapse of German payments group Wirecard in a high-profile fraud.

“AI tools are now moving beyond experimentation to becoming a reality in certain audit scenarios,” said Mark Babington, FRC executive director of regulatory standards. 

KPMG UK has begun to use AI tools for sophisticated audit techniques, including AI transaction scoring — it scans millions of data transactions to identify those of most note to the auditor, according to Emily Jefferis, head of audit quality, something she said was not possible via traditional techniques.

Meanwhile Deloitte’s audit teams use AI to summarise board minutes, extract information from complex contracts, and to streamline other manual processes, according to a person familiar with the matter.

Because of its rapid adoption, the FRC has encouraged firms to define metrics to evaluate the impact of AI tools on audit quality. “The use of [automated tools] has significant potential to improve audit quality, though this is dependent on the [tools] producing consistently reliable outputs and being used routinely in the intended manner,” the report said.

Since the review began, the regulator said that firms had begun to modernise their oversight in this area.

KPMG UK’s Jefferis said however that quantifying the impact of such tools was a “subjective matter”. She said: “We carefully monitor the adoption of all our tools using a range of KPIs and have the aim of putting AI in the hands of every auditor for use in every engagement. We are currently close to that target.”

The FRC’s findings come as the Big Four race to devise a new kind of audit that would assess the effectiveness of clients’ own AI tools. The audits could open a revenue stream for auditors, similar to the demand for assurance for companies’ environmental, social and governance metrics.