Can AI Protect Client Confidentiality?

The problem, in numbers

Professional firms hold tax files, deal documents, family disputes and company accounts, and the business rests on clients trusting that it stays put. Breaches already happen without AI. The Office of the Australian Information Commissioner received 1,205 data breach notifications in 2025, the most since the scheme began in 2018, and 81 of them came from legal, accounting and management services.^{Source 1} In the second half of 2024, 170 of 595 notifications were put down to human error, and the largest human-error cause was personal information emailed to the wrong person.^{Source 1}

Generative AI adds a new route for human error: pasting client material into a tool the firm does not control. A University of Melbourne and KPMG survey of 48,340 people in 47 countries found almost one in two employees who use AI admit to doing so in ways that breach organisational policy. About half (48 to 49%) report uploading sensitive company information, such as financial, sales or customer information, or copyrighted material into public AI tools.^{Source 2} These are global figures, and the authors note that even an anonymous survey may understate the behaviour.^{Source 2}

What improves, and why

The question here is not whether AI can guard client data, but whether a firm can use AI without giving that data away. That depends less on the model than on how it is supplied.

Guidance from the Australian Signals Directorate, written with partner agencies in ten other countries, sets out the mechanism plainly. Information given to a generative AI system may be incorporated into its training data and could inform answers given to people outside the organisation. Before using a third-party system, a firm should find out whether its inputs are used to retrain the model, consider a private version, check the system meets data residency obligations, know what happens to its data when the agreement ends, and train staff on what must never be entered.^{Source 6}

A firm that provides an approved tool, under terms it has read, gives staff a sanctioned route for work they already do with AI. In our reading, that is the strongest case for safety here: not AI as a protector, but a governed tool replacing an ungoverned one.

The alternative, banning AI, does not appear to work. In the same global survey, policy breaches and uploads to public tools were most commonly reported by employees whose organisation had banned generative AI (67%), against 33% where there was no policy.^{Source 2} The authors suggest bans may be ineffective.^{Source 2}

What the evidence actually says

In 2023 Samsung staff reportedly uploaded sensitive code to ChatGPT, and the company temporarily restricted generative AI on company computers. In a company-wide survey the month before, 65% of respondents had said they were concerned about security risks from these services.^{Source 3}

The Australian case is closer to professional practice. A Victorian child protection worker entered personal and sensitive case details into ChatGPT while drafting a report for the Children's Court. The Victorian Information Commissioner found the information had been disclosed outside the department's control and that the report contained inaccurate personal information which downplayed the risks to the child.^{Source 4} A review of one year of the unit's cases found 100 with indicators of ChatGPT use, nearly 900 staff across the department had accessed the ChatGPT website in the second half of 2023, and staff had not been trained on how these tools work or their privacy risks. The regulator ordered the department to bar its child protection staff from public generative AI tools and block access to them.^{Source 4} The disclosure and the quality failure came together, and both were found after the fact, not stopped by a control.

Regulators and professional bodies now draw the same line. The OAIC says privacy obligations apply to any personal information entered into an AI system, and recommends that organisations do not enter personal information, particularly sensitive information, into publicly available generative AI tools.^{Source 5} The Law Society of NSW and the legal regulators of Western Australia and Victoria state that lawyers "cannot safely enter confidential, sensitive or privileged client information into public AI chatbots", and that commercial tools need their contract terms checked.^{Source 7} The Tax Practitioners Board goes furthest: its July 2026 guidance says entering client information into an AI tool can be a disclosure to a third party, depending on how the tool is configured and used, which requires each client's permission. It recommends telling clients where their data will be stored and whether AI may be used, and permission can be given in the engagement letter.^{Source 8}

We found no study comparing confidentiality incidents in firms using governed AI tools with firms that do not. The case rests on regulatory logic and documented failures, not measured results.

Where it falls short

• The setting is invisible to the user. A consumer account and an enterprise deployment can look much the same on screen. Whether an entry is a disclosure depends on configuration and use,^{Source 8} which the person typing rarely sees.

• Bans move the risk, not remove it. Policy breaches were most commonly reported where AI had been banned outright.^{Source 2}

• Privacy law covers part of the problem. The Privacy Act protects personal information. In our reading, a client's commercial secrets and legal privilege depend on professional duties and contract terms, so a privacy check alone is not a confidentiality check.

• Due diligence goes stale. Vendor terms change, and the OAIC says due diligence on an AI product should not be "set and forget".^{Source 5}

Where governance decides the outcome

The legal stakes are rising. The Privacy and Other Legislation Amendment Bill, passed in November 2024, brought new tiers of civil penalties, infringement notices and a statutory tort for serious invasions of privacy, and the Privacy Commissioner called it "an important first step".^{Source 5} A firm that wants AI's efficiency on client work needs four things settled in writing before anyone uses it: an approved tool list, with contract terms covering training on inputs, data location and what happens to data when the agreement ends;^{Source 6} client permission that says where data goes and whether AI is used;^{Source 8} staff training on what must never be entered;^{Source 6} and a named owner who reviews the arrangement as vendors change their terms.

How to think about adoption

Start with an inventory of the AI tools staff already use, including on their own accounts. Then choose a contained tool for client work and read its terms against the cyber agency's questions: training on inputs, residency and exit. Update engagement letters so clients know whether AI is used and where their data goes, as the Tax Practitioners Board now recommends. Train people on a short list of what never goes in. Keep public tools for work with no client information, rather than banning them and driving use out of sight. Only then count the time saved. An efficiency that depends on a disclosure the client never agreed to is not a saving the firm can keep.