Finance teams should build governance into AI operating models

“Governance cannot be something added around AI after deployment; it needs to be built into the operating model from the outset,” Cairnes said.

Cairnes encouraged chief financial officers (CFO) to establish a clear framework for how AI can operate within finance, including what data it could access, what processes it could act on, where human judgement would be required, and how every action can be tracked and explained.

“The standard should be the same that finance already applies to its own processes,” she said

“For example, can we understand what happened, why it happened and who was accountable for the outcome? If finance cannot see inside an AI system, it cannot build the confidence and trust at scale.”

Australian businesses are taking a structured approach into AI governance, including undertaking a risk-based assessment, accountability, and human oversight.

When asked how finance teams could implement these principles in practice when they use AI for core accounting and financial processes, Cairnes recommended making governance proportionate to the level of risk and judgement involved.

While routine, AI could be used to automate routine, rules-based work, Cairnes said exceptions, approvals, and decisions with a significant financial impact would require “defined” human intervention.

“The distinction between black-box and glass-box AI becomes critical here. Finance teams need to be able to see what data an AI system used, how it reached a conclusion and what controls were applied before an action was taken. Every step needs to be auditable and explainable,” Cairnes said.

“That is what turns AI governance from a policy into a working control environment: clear boundaries around what AI can do, defined points for human intervention and a complete, auditable record from transactions through to financial reporting.”

When asked about the biggest governance mistakes organisations make when they scale AI in finance, Cairnes said it is “scaling probabilistic AI in finance without a deterministic control layer and unified data foundation”.

“The result is a system that looks advanced on the surface but fails the most basic finance test: can you prove, with evidence, that the outcome is right?” she said.

“AI’s role should be earned, not assumed. Start with processes where the risk is understood, prove that the technology is transparent and reliable, then expand its autonomy as confidence grows.”

Cairnes noted that many organisations may fall into the temptation of measuring AI success by how many tasks and functions are automated, but cautioned them against this approach.

“The real test is whether AI enables finance to increase its impact on the business while maintaining the accountability and trust on which the function depends,” she concluded.