AI Governance for Australian Organisations — What You Need to Know

Something changed — and most organisations missed it

Between late 2022 and early 2024, something fundamental shifted in every Australian workplace. AI tools ChatGPT, Microsoft Copilot, Google Gemini, Claude, and dozens of others became available to anyone with a web browser and an email address. No procurement process. No IT approval. No security review. Just a browser tab and a conversation.

The adoption was immediate and largely invisible. A solicitor pastes a client's confidential settlement terms into ChatGPT to help draft a response letter. A GP uses an AI tool to summarise a complex referral. An accountant uploads a client's financial statements to an AI assistant for analysis. A claims handler pastes policyholder details into an AI tool to draft a decision letter. A recruiter uploads candidate CVs to an AI platform for screening.

In every one of these cases, the person using the AI tool is trying to do their job more efficiently. They are not being reckless. They are not deliberately breaching policy. They are using a tool that appeared in their browser, that their colleagues are using, that the media is celebrating as a productivity revolution. The tool works. It makes them faster. So they use it.

The problem is not the intent. The problem is what happens to the data.

When sensitive information patient records, privileged legal communications, client financial data, personal information, trade secrets enters an AI tool, it leaves the organisation's control. The AI provider's terms of service determine what happens next. In many cases, the data may be used to train models, may be stored indefinitely, may be accessible to the provider's staff, and may be subject to foreign jurisdiction. The organisation that was responsible for protecting that data has, in a single browser interaction, lost its ability to prove where the data went and what happened to it.

This is happening in Australian organisations right now. Every day. At scale. And most organisations have no visibility into it.

The shadow AI problem

"Shadow AI" is the term used to describe AI tool usage that occurs without organisational knowledge, approval, or governance. It is the AI equivalent of shadow IT but faster, more pervasive, and harder to detect.

Shadow AI is different from shadow IT in three important ways.

First, the barrier to entry is zero. Shadow IT typically involved someone installing unapproved software or spinning up an unapproved cloud service. There was a technical step a download, an installation, a subscription that IT could potentially detect. Shadow AI requires nothing more than opening a browser tab. There is no software to install, no subscription to create (for free tiers), and no network signature that distinguishes "person browsing a website" from "person pasting client data into an AI model."

Second, the data exposure is immediate and complete. When someone uses an unapproved cloud storage service (shadow IT), the data is stored in a specific location and can potentially be retrieved or deleted. When someone pastes data into an AI tool, the data enters a processing pipeline that the organisation has no visibility into, no control over, and in most cases, no contractual relationship with. The data may be processed, stored, used for model training, or retained in ways the organisation cannot verify or reverse.

Third, the usage is rational. Shadow IT was often a workaround for inadequate tools — people used Dropbox because the company file share was slow. Shadow AI is not a workaround. It is a productivity tool that genuinely makes people better at their jobs. A lawyer who uses AI to draft a research memo is not circumventing a policy out of frustration; they are using the most effective tool available. This makes shadow AI harder to address because the solution cannot simply be "stop using AI." The solution has to be "use AI, but with governance."

The scale of shadow AI in Australian organisations is difficult to measure precisely because, by definition, it is invisible to the organisation. However, surveys consistently show that between 50% and 80% of knowledge workers in developed economies have used AI tools at work, and a significant majority of that usage occurs without formal organisational approval or governance.

For Australian organisations that handle sensitive data, healthcare providers, law firms, accounting practices, financial services firms, insurers, and government agencies, shadow AI is not a theoretical risk. It is a current, ongoing, measurable data governance failure.

What the law says today — and where it is heading

Australian law does not yet have a comprehensive AI-specific governance framework. However, existing laws already impose obligations that directly affect how organisations use AI with sensitive data.

The Privacy Act 1988 and Australian Privacy Principles

The Privacy Act applies to most Australian organisations with an annual turnover of more than $3 million, as well as healthcare providers, government agencies, and organisations that trade in personal information (regardless of turnover).

Australian Privacy Principle 6 (APP 6) restricts the use and disclosure of personal information. An organisation that collects personal information for a specific purpose (such as providing medical care or legal services) is generally prohibited from using or disclosing that information for a different purpose without the individual's consent. When an employee pastes personal information into an AI tool, that is a disclosure to the AI provider, a third party, for a purpose that the individual almost certainly did not consent to.

Australian Privacy Principle 8 (APP 8) restricts cross-border disclosure of personal information. Most AI tools are operated by companies in the United States. When personal information enters these tools, it is disclosed to an overseas recipient. APP 8 requires the organisation to take reasonable steps to ensure the overseas recipient handles the information consistently with the APPs, a condition that is essentially impossible to meet when the organisation has no contractual relationship with the AI provider and no visibility into its data handling practices.

Australian Privacy Principle 11 (APP 11) requires organisations to take reasonable steps to protect personal information from misuse, interference, loss, unauthorised access, modification, and disclosure. Allowing employees to paste personal information into uncontrolled AI tools, without monitoring, without policy enforcement, and without evidence of what occurred, is difficult to reconcile with the "reasonable steps" requirement.

The Notifiable Data Breaches scheme

Under Part IIIC of the Privacy Act, organisations must notify the OAIC and affected individuals when a data breach involving personal information is likely to result in serious harm. If personal information enters an AI tool and the organisation cannot determine what happened to it, whether it was stored, used for training, accessed by the provider's staff, or retained indefinitely, the question of whether a notifiable breach has occurred becomes complex and uncomfortable.

Most organisations have not considered whether uncontrolled AI usage constitutes a potential notifiable data breach. The answer depends on the specific AI tool's terms of service, the type of data involved, and whether the exposure is likely to cause serious harm. But the threshold for "likely to result in serious harm" is not high when the data involved is medical records, legally privileged communications, or financial information.

Industry-specific obligations

Beyond the Privacy Act, industry-specific regulations impose additional obligations:

Healthcare organisations are subject to the My Health Records Act 2012, RACGP information security standards, and the Healthcare Identifiers Act 2010. Patient data entering AI tools without governance may breach these obligations.

Legal practitioners are subject to Australian Solicitors Conduct Rules requiring the protection of client information and legal privilege. Privileged communications pasted into AI tools may constitute a waiver of privilege a consequence that cannot be undone.

Financial services firms and insurers regulated by APRA are subject to CPS 234, which requires information security capabilities commensurate with the size and extent of threats. Uncontrolled AI usage with policyholder or customer data is an information security threat that CPS 234 requires the organisation to manage.

The regulatory direction

The Australian Government has signalled its intention to develop a more comprehensive AI governance framework. The consultation process has included principles-based approaches, risk-based classification of AI systems, and sector-specific guidance. While the timeline and final form of Australian AI regulation remain uncertain, the direction is clear: organisations will be expected to demonstrate how they govern AI use, what controls they have in place, and what evidence they can provide.

The organisations that establish AI governance now, with monitoring, enforcement, and evidence, will be well-positioned when regulatory requirements formalise. Those who wait will face a more difficult and more expensive transition.

Why policies alone are not governance

Most organisations that have taken any action on AI governance have written a policy. The policy typically says something like: "Employees must not enter sensitive or confidential information into AI tools without approval." Some policies are more detailed, listing specific AI tools that are approved, specifying what types of data may and may not be used, and requiring employees to complete training.

Writing a policy is a necessary step. It is not sufficient.

A policy that is not enforced is not governance. It is documentation. The distinction matters for three reasons.

First, an unenforced policy does not prevent data exposure. A policy in a handbook or an intranet page does not stop a clinician from pasting patient data into ChatGPT at 11 pm when they are trying to draft a referral letter. The clinician may not remember the policy. They may not have read it. They may have read it, but consider this particular use to be low-risk. The policy exists. The data exposure happens anyway.

Second, an unenforced policy is difficult to defend in a regulatory or legal context. If an organisation suffers a data breach through uncontrolled AI usage and the OAIC investigates, the organisation's defence that "we had a policy" will be weakened by the follow-up question: "What steps did you take to enforce the policy?" If the answer is "we told people about it in onboarding," that is unlikely to satisfy the "reasonable steps" requirement of APP 11. A reasonable step is a control that actually prevents or detects the behaviour, not a document that asks people not to do it.

Third, an unenforced policy produces no evidence. Even if no breach occurs, the organisation has no evidence of compliance. It cannot demonstrate to a board, an insurer, a regulator, or a client that its AI governance is working because it has no data about what is actually happening. It has a policy document that says what should happen. It has no evidence of what is happening.

Real AI governance requires three things that a policy alone cannot provide: visibility into how AI tools are actually being used across the organisation, enforcement of the organisation's rules at the point of AI interaction, and evidence that the governance system is working.

What real AI governance looks like

Effective AI governance operates at the point of interaction where the employee meets the AI tool and produces evidence of what happened and what was done about it. It has five components.

Component 1: Discovery — knowing what AI is being used.

Before an organisation can govern AI use, it needs to know what AI tools are being accessed, by whom, from which devices, and for what purposes. This includes both sanctioned and unsanctioned tools. Most organisations have no visibility into AI tool usage beyond what employees self-report, which, given the nature of shadow AI, means they have almost no visibility at all.

Effective discovery monitors AI-related network traffic, browser activity, API calls, and application usage to build a comprehensive picture of how AI tools are being used across the organisation. The output is an inventory not of what the organisation thinks is happening, but of what is actually happening.

Component 2: Classification — understanding what data is involved.

Not all AI interactions carry the same risk. An employee using ChatGPT to brainstorm marketing taglines is a different risk profile from a lawyer pasting privileged communications into the same tool. Effective governance distinguishes between these interactions based on the sensitivity of the data involved.

Classification examines the content of AI interactions what is being sent to the AI tool, and what is coming back and determines the data sensitivity level. This might involve pattern-based detection (identifying tax file numbers, credit card numbers, health identifiers), semantic analysis (identifying privileged legal content, medical records, financial advice), and contextual assessment (who is the user, what is the destination tool, what is the organisational policy for this combination).

Component 3: Enforcement — applying the organisation's rules automatically.

Once an AI interaction is detected and the data sensitivity is classified, the governance system applies the organisation's policy. The available actions typically include:

Blocking the interaction entirely prevents the data from reaching the AI tool. Appropriate for highly sensitive data being sent to unsanctioned tools.

Warning the user, allowing the interaction to proceed, but displaying a notification that explains the risk and asks the user to confirm. Appropriate for moderate-risk interactions where the user may not have considered the sensitivity of the data.

Redacting sensitive content automatically removes or masks sensitive data elements from the content before it reaches the AI tool. Appropriate where the AI tool is approved, but certain data types should not be included.

Recording for review, allowing the interaction to proceed but flagging it for compliance review. Appropriate for interactions that are within policy but warrant monitoring.

Allowing without intervention permitting the interaction to proceed with standard evidence recording. Appropriate for low-sensitivity interactions with approved AI tools.

The critical principle is that enforcement happens automatically, at the point of interaction, without requiring the user to self-assess the risk. The user's behaviour triggers the governance system. The governance system applies the rules. The user does not need to remember the policy, interpret the policy, or decide whether the policy applies to this particular interaction.

Component 4: Evidence — recording what happened.

Every governed AI interaction should produce an immutable evidence record: what data was involved, who was the user, what AI tool was the destination, what classification was applied, what policy action was taken, and when it happened. This evidence should be tamper-proof, written to storage that cannot be altered after the fact and linked to the organisation's overall evidence infrastructure.

This evidence serves multiple purposes. It demonstrates to the board that AI governance is operational. It demonstrates to insurers that AI-related risks are being managed. It demonstrates to regulators that the organisation is taking "reasonable steps" to protect personal information. And if an incident occurs, it provides a forensic record of what happened, enabling investigation rather than guesswork.

Component 5: Integration — working with existing security tools.

Most organisations already have security tools, a SIEM for security event management, an EDR platform for endpoint detection, a SASE or web proxy for network traffic, and an ITSM system for incident management. AI governance should not require replacing these tools. It should integrate with them.

When an AI governance system detects a high-severity policy violation, it should automatically create an incident in the organisation's ITSM system, send an alert to the SIEM, and notify the relevant team through the organisation's existing communication channels. This integration ensures that AI governance events are visible within the security infrastructure the organisation already operates not siloed in a separate system that nobody monitors.

The evidence question — can you prove your governance is working?

Of the five components above, evidence is the one that most organisations overlook even those that have some level of AI monitoring in place.

Having a monitoring dashboard that shows AI interactions is useful for operational visibility. But it is not the same as having an immutable evidence trail that can be presented to a board, an insurer, a regulator, or a court.

Consider the difference:

A monitoring dashboard says: "We can see AI usage in real time." If someone asks "what happened on 15 March?" the answer depends on whether the logs were retained, whether they were accessible, and whether they can be verified as genuine.

An evidence infrastructure says: "Every governed AI interaction is recorded as an immutable, timestamped, cryptographically hashed evidence record that cannot be altered. We can produce a verified trail of what happened on 15 March or any other date, with proof that the evidence has not been tampered with since collection."

The second statement is defensible in a regulatory investigation, an insurance claim, a board inquiry, or a legal proceeding. The first is not, because logs without integrity guarantees are just data. They can be questioned, disputed, or dismissed.

For Australian organisations that handle sensitive data, the evidence question is not academic. The Privacy Act requires "reasonable steps." The Notifiable Data Breaches scheme requires organisations to assess the impact of breaches. Industry regulators require demonstrable security capabilities. Boards require defensible governance reporting. In all of these contexts, "we had a dashboard" is weaker than "here is the evidence, and you can verify it independently."

Who needs AI governance and who can wait

Not every organisation faces the same level of AI governance urgency. The priority depends on the sensitivity of the data the organisation handles, the regulatory environment it operates in, and the likelihood that employees are using AI tools with sensitive data.

High urgency — act now:

Healthcare organisations: patient data, My Health Records Act obligations, RACGP standards. Clinical and administrative staff are using AI tools for referral letters, clinical summaries, patient communications, and administrative tasks. The data involved is among the most sensitive categories under Australian law.

Legal practices: privileged communications, confidentiality obligations across all jurisdictions, client files. Lawyers and support staff are using AI for research, drafting, document review, and case analysis. Loss of privilege through AI disclosure may be irrecoverable.

Financial services and insurance: policyholder data, APRA CPS 234 obligations, financial information. Claims handlers, underwriters, and analysts are using AI tools for data analysis, correspondence, and decision support.

Government agencies: citizen data, ASD ISM obligations, security classification requirements. Government employees are using AI tools for policy drafting, analysis, and communications.

Moderate urgency — plan and begin:

Accounting firms: client financial data, tax file numbers, business-sensitive information. Staff are using AI for analysis, report drafting, and tax research.

Professional services: client confidential information, project data, strategic documents. Consultants and analysts are using AI for research, presentations, and analysis.

MSPs and IT providers: access to multiple clients' data and systems. Staff may use AI tools while working within client environments, exposing client data through the MSP's AI usage.

Lower urgency — but do not ignore:

Organisations that handle primarily non-sensitive data or that have very small teams may have lower immediate risk from AI usage. However, every organisation that handles personal information (which includes employee personal information) has Privacy Act obligations that AI usage can affect. No organisation is exempt from the need to understand how AI tools are being used in its environment.

Practical steps — what to do this quarter

If your organisation has not yet addressed AI governance, or has addressed it only through policy, here are practical steps you can take in the next 90 days.

Step 1: Understand what is happening today.

Before writing or updating policies, find out what AI tools your people are actually using. This does not require sophisticated technology it can start with a confidential survey asking employees which AI tools they use, how often, and for what purposes. The results will almost certainly surprise you. The gap between "what we think is happening" and "what is actually happening" is significant in every organisation we have seen.

Step 2: Identify your highest-risk data flows.

Based on what you learn from the survey (and ideally from technical discovery tools), identify the data flows with the highest risk. These are the interactions where the most sensitive data is being sent to the least-controlled AI tools. Focus your initial governance efforts on these high-risk flows rather than trying to govern everything at once.

Step 3: Write or update your AI usage policy — but do not stop there.

If you do not have an AI usage policy, write one. If you have one, review it against what you learned in steps 1 and 2. Make sure the policy is specific enough to be enforceable: which tools are approved, which data types may and may not be used, what approval process exists for new tools, and what happens when someone violates the policy.

But remember: writing the policy is necessary but not sufficient. The policy needs an enforcement mechanism. Without enforcement, the policy is a document, not governance.

Step 4: Evaluate enforcement options.

Depending on the urgency and the sensitivity of your data, evaluate options for enforcing your AI policy at the point of interaction. This might include browser-based monitoring, endpoint-level controls, network-level inspection, or API-level interception. The right approach depends on your organisation's size, technology environment, risk profile, and budget.

Platforms like APEXLyn Lens are designed specifically for this purpose, providing multiple enforcement layers (browser, endpoint, network, API, cloud connectors) that work alongside your existing security tools and produce forensic-grade evidence of every governed AI interaction.

Step 5: Start collecting evidence from day one.

Whatever governance approach you adopt, make sure it produces evidence. Not just logs, immutable, timestamped, verifiable evidence that proves your governance is working. This evidence will be valuable for board reporting, insurance discussions, regulatory responses, and, if an incident occurs, forensic investigation.

The organisations that start collecting AI governance evidence now will be in a fundamentally stronger position than those that wait for regulation to force them to start. By the time regulatory requirements are formalised, early movers will have months or years of evidence demonstrating responsible AI governance. Late movers will be scrambling to implement controls and will have no historical evidence to show.

The bottom line

AI governance is not about restricting AI use. It is about enabling AI use responsibly with visibility into what is happening, enforcement of organisational rules, and evidence that governance is working.

The organisations that get this right will be able to use AI confidently, knowing that their data is protected, their obligations are met, and their governance is provable. The organisations that ignore it or address it only with a policy document are accumulating risk that will eventually be realised, whether through a breach, a regulatory action, a disputed insurance claim, or a board-level discovery that the organisation's AI usage has been uncontrolled all along.

The technology to govern AI use exists today. The question is not whether your organisation will need AI governance. The question is whether you will implement it before or after something forces you to.