iscoverA recent Deloitte report exposed the dangers of unchecked AI, highlighting the need for critical human oversight and careful data validation when Ai is incorporated into workflows.
Dangers of AI Hallucinations: The recent Deloitte Report
13 September 2025
•
5 minute read
Artificial intelligence is rapidly changing the accounting landscape, but a recent incident involving a Deloitte Report serves as a stark reminder of the potential pitfalls. The report, commissioned by the Department of Employment and Workplace Relations (DEWR) for $440,000, was found to contain significant errors, including fabricated academic references and a false quote attributed to a Federal Court judge.
Dr. Christopher Rudge, a sociolegal scholar at the University of Sydney, uncovered the errors, revealing instances of non-existent books, papers, and misattributed legal statements. Deloitte admitted to using AI in producing the report and partially refunded the fee.
The Roots of the Problem: AI and Hallucinations
You’re out of free articles for this month
To continue reading the rest of this article, please log in.
Create free account to get unlimited news articles and more!
The errors in the Deloitte report highlight a core challenge with AI: its propensity for "hallucinations," where it generates incorrect or fabricated information. Ian Youngman, managing Director and co-founder of AI accounting service provider Elfworks, believes understanding the underlying characteristics of AI models is crucial to mitigating this risk.
“AI models learn from vast datasets, identifying patterns and predicting the next word or phrase in response to a query. They don't "understand" the world like humans do and are more likely to provide answers, even if they are inaccurate, than to admit ignorance” says Youngman. “Furthermore, most AI models lack a live database to validate output. This can lead to the creation of plausible but entirely fictional content.”
James McPhedran, co-founder of Elfworks, believes people learn best through storytelling. He’s crafted a story to illustrate the AI hallucination problem at the core of the Deloitte Report.
Chat GPT vs The Librarian
“Let’s assume Deloitte asked a librarian to find academic articles or books relevant to the Report for the DEWR”, says McPhedran. “The librarian searches the full content of a library and fails to find academic articles or books directly on point (as there are no such articles or books directly on point in the library) yet there are some articles loosely related to the content of the Report. After 5 hours of research, the librarian reports these findings back to Deloitte.
At the same time, Deloitte asked an AI model (that has been trained up on the entire content of the library) to find academic articles or books relevant to the Report for the DEWR. The AI model fails to find an academic article or book directly on point (as there are no such articles or books – the librarian was right) however as it wants to please Deloitte, it mashes together the name of an author it has read within the content of the library with other words and phrases from the content of the library relevant to the query and author and creates a fictional book written by the aforementioned real author. Every word within the book – and the author’s name – exist within the library, they just do not exist in the form we refer to as a ‘book’ or ‘article’; it’s a mash-up of other books and articles.”
The Path Forward
The Deloitte report serves as a valuable lesson for the accounting profession. While AI offers significant potential, it must be used responsibly and with a critical eye.
If you would like to trial a platform designed to help Australian accountants in public practice with research, advice drafting and a range of productivity tools that has built-in databases and validation steps to weed-out hallucinations, please contact us on
You need to be a member to post comments. Become a member for free today!
