HIPAA and AI Tools: What Healthcare Organizations Need to Know

May 19, 2026 By ShadowLock Team HIPAAhealthcareAI compliancePHI

Using AI tools with protected health information without a Business Associate Agreement is a HIPAA disclosure issue. When a clinical staff member pastes a patient case summary into ChatGPT, the PHI has now been disclosed to a third party with no BAA, no documented safeguards, and unclear data retention. Even a single such paste can trigger breach notification obligations. Below is a plain-English guide for healthcare organizations and their MSPs on what HIPAA requires for AI tool use, and how to build a compliant program.

The HIPAA-AI intersection is one of the most consequential gaps in healthcare IT in 2026. The volume of incidents we see in customer telemetry is significant, and the regulatory attention is catching up.

The Core HIPAA Issue with AI Tools

HIPAA defines two categories of organizations: covered entities (healthcare providers, health plans, healthcare clearinghouses) and business associates (vendors who handle PHI on behalf of covered entities). Both categories have specific obligations under the Privacy Rule and Security Rule.

The issue with consumer AI tools: they are not business associates of your organization. They have not signed a BAA. Their standard terms typically do not satisfy HIPAA’s requirements for safeguarding PHI. When an employee pastes PHI into ChatGPT, Claude, or Gemini consumer accounts, the organization has disclosed PHI to a non-BAA third party, which is a HIPAA disclosure under 45 CFR §164.502.

The disclosure happens even if:

The employee did not intend to share PHI
Nothing bad happens downstream
The AI tool does not retain the data
The disclosure was internal-feeling (a single chat with no human reader)

The framework cares about the disclosure, not the downstream consequences.

What Counts as PHI in AI Tool Use

The PHI categories that most often appear in AI tool pastes:

Patient names combined with any health information
Medical record numbers (MRNs)
Diagnosis codes (ICD-10, CPT codes) combined with patient identifiers
Insurance information (member IDs, policy numbers)
Date of birth combined with health information
Patient demographics in case summaries
Provider notes referencing specific patients
Lab results with patient identifiers
Imaging study identifiers combined with findings

The 18 HIPAA identifiers when combined with health information all qualify as PHI. Pasting any of them into a non-BAA AI tool is a disclosure event.

The Safeguards HIPAA Expects

The HIPAA Security Rule (45 CFR §164.302-318) defines administrative, physical, and technical safeguards. Several apply directly to AI tool use:

§164.308(a)(1), Security Management Process

Requires risk analysis and risk management. As of 2025-2026, AI tool use is increasingly expected to appear in healthcare risk assessments. Organizations that have not assessed AI as a risk vector are starting to be flagged on HHS audits.

§164.308(a)(4), Information Access Management

Requires policies and procedures for authorizing access to ePHI. AI tools accessing ePHI without authorization (a BAA) is a clear gap. Solving this requires either (a) authorizing the AI tool via BAA, or (b) preventing the access technically.

§164.312(a)(1), Access Control

Technical safeguard requiring access control mechanisms. For AI specifically, this means: technical controls that prevent unauthorized AI tools from receiving ePHI. The endpoint-and-browser layer AI DLP is the practical implementation.

§164.312(b), Audit Controls

Technical safeguard requiring audit logs of system activity involving ePHI. AI tool events involving PHI must be in the audit log. Per-event records, user, timestamp, tool, PHI classifier match, outcome, are what HIPAA auditors increasingly examine.

§164.312(c)(1), Integrity Controls

Requires controls that protect ePHI from improper alteration or destruction. Less directly applicable to AI tool use, but the block-and-log pattern provides supporting evidence.

§164.308(a)(5), Security Awareness and Training

Requires training on security practices. AI policy training and acknowledgement records are part of this evidence package.

The HIPAA AI Compliance Path

The compliant path for healthcare organizations using AI:

Option 1: BAA with the AI Vendor

Some AI vendors offer enterprise tiers that include a BAA. Microsoft Copilot for M365 includes a BAA at the enterprise tier for some configurations. Google Workspace AI features have similar enterprise options. OpenAI’s enterprise offerings can include BAAs for specific deployments.

If you BAA with the vendor, you can use the tool with PHI provided you stay within the BAA’s scope, which is typically the enterprise-tier deployment, not the consumer-tier.

Option 2: Technical Controls Preventing Disclosure

If you do not BAA (or for consumer-tier tools you cannot BAA), you need technical controls preventing PHI from reaching the tool. This is the AI DLP path: a content classifier detects PHI patterns at the moment of paste and blocks the submission.

Most healthcare organizations need both. They BAA with their primary enterprise AI tool (typically Microsoft Copilot or Google Workspace AI) for sanctioned use, and they deploy AI DLP across the fleet to prevent disclosure to non-BAA tools (consumer ChatGPT, Claude, Gemini, etc.).

The Healthcare-Specific Risk Patterns

The patterns we see in healthcare customer telemetry:

Clinical staff drafting communications

The single most common pattern. A nurse or physician pastes a case summary, including patient name, diagnosis, and treatment plan, into ChatGPT to draft a referral letter or a patient-facing explanation. The intent is benign (faster, clearer communication); the impact is HIPAA disclosure.

Billing and revenue cycle teams

A billing specialist pastes a denied claim, including patient information, diagnosis codes, and the insurance reason for denial, into an AI tool to help draft an appeal letter. Multiple HIPAA identifiers in a single paste.

Behavioral health and mental health staff

Behavioral health information is among the most sensitive PHI categories. Staff drafting therapy notes, treatment plans, or crisis communications using AI tools represents some of the highest-severity risk events in healthcare.

Research and informatics

Research staff pasting de-identified datasets into AI tools for analysis, but the de-identification is often incomplete (small geographic areas, dates of service, rare diagnoses). What looks de-identified to the user may still contain re-identifiable PHI.

Front office and administrative

Front office staff pasting scheduling information, billing inquiries, or patient communications into AI tools to draft responses. Lower per-event severity but very high volume.

Building the HIPAA-Compliant AI Program

The practical sequence for a healthcare organization:

Phase 1: Risk Assessment (Week 1-2)

Add AI tool use to your formal HIPAA risk assessment. Document the categories of staff likely to use AI, the data they handle, and the current state of controls. This satisfies §164.308(a)(1).

Phase 2: Policy (Week 1-2, parallel)

Publish an AI acceptable use policy that explicitly addresses PHI. Our free policy template includes PHI-specific language. Customize for your organization, have legal and compliance review, distribute.

Phase 3: Sanctioned AI Tool (Week 2-4)

If your organization needs AI for clinical or operational use, license a BAA-covered enterprise AI tool. Microsoft Copilot for M365 (with the appropriate enterprise tier) and Google Workspace AI features are common choices. Document the BAA in your business associate inventory.

Phase 4: Technical Controls (Week 3-6)

Deploy AI DLP with the PHI classifier enabled. Start in monitor-only mode for two weeks to baseline activity. Then promote to blocking on PHI submissions to non-BAA AI tools.

Phase 5: Training (Week 4-8)

Roll out AI policy training to all clinical and administrative staff. Collect acknowledgements via your HRIS. Satisfies §164.308(a)(5).

Phase 6: Audit Logging (Configured during Phase 4)

Confirm the AI DLP platform produces audit logs with retention aligned to your HIPAA program (often 6 years for compliance evidence). Satisfies §164.312(b).

By the end of two months, most healthcare organizations have a working HIPAA-compliant AI program.

What Healthcare MSPs Need to Know

MSPs serving healthcare clients have specific obligations. The MSP is often a business associate of the healthcare client and shares responsibility for safeguards.

For MSP-served healthcare clients:

The MSP needs a BAA with the healthcare client (standard)
The MSP must deploy controls consistent with the client’s HIPAA program
The MSP’s AI governance platform must satisfy HIPAA technical safeguards
Per-tenant audit log isolation matters, each healthcare client’s logs are separate

ShadowLock’s multi-tenant AI governance for MSPs is purpose-built for this. Each client tenant has isolated PHI classification, blocking policy, and audit logs.

Frequently Asked Questions

Is using ChatGPT a HIPAA violation?

It depends on the context. Using consumer ChatGPT with PHI is a HIPAA disclosure issue (no BAA). Using ChatGPT Enterprise with PHI under a BAA can be compliant. Using ChatGPT without any PHI is not a HIPAA issue regardless of tier.

Does Microsoft Copilot for M365 cover HIPAA?

The enterprise tier of Microsoft Copilot for M365 includes a BAA under specific configurations. Confirm with Microsoft that your specific deployment includes the BAA, terms vary by tier, region, and license type.

Can I use ChatGPT for clinical documentation?

Only if it is ChatGPT Enterprise with a BAA, used within the BAA’s scope. Consumer ChatGPT is not appropriate for clinical documentation under any HIPAA-covered scenario.

What about behavioral health information?

Behavioral health PHI is subject to additional protections (42 CFR Part 2 in some cases). The HIPAA rules apply plus additional restrictions. Behavioral health is typically the most sensitive category in healthcare and warrants the strictest AI tool controls.

Are de-identified datasets safe to use with AI tools?

Only if the de-identification meets the HIPAA standards (Safe Harbor or Expert Determination methods, 45 CFR §164.514). Incomplete de-identification can still contain re-identifiable PHI. Many “de-identified” datasets that look anonymized are not de-identified by HIPAA standards.

What if a HIPAA disclosure to an AI tool happens?

Investigate as you would any potential PHI disclosure. Determine the scope (which patients, which data), the risk of harm, and whether breach notification applies under 45 CFR §164.404. Document the investigation. AI tool disclosures may or may not trigger notification depending on the specific facts.

How does the technical control help during a HIPAA audit?

HHS auditors increasingly examine technical safeguards for AI tool use. The AI DLP platform’s audit log, showing PHI submissions detected and blocked, with retention aligned to HIPAA, directly satisfies §164.312(a)(1) and §164.312(b) for the AI tool vector. Without the technical control, the audit answer is uncomfortable.

HIPAA and AI tools are now a primary intersection of healthcare compliance. The combination of BAA-covered enterprise AI for sanctioned use plus AI DLP for unsanctioned use is the working compliance pattern in 2026. Healthcare organizations that build it now will be ready for the HHS attention that is coming.