Shadow AI Incidents: A Running List for MSPs

Reported breach Dec 17, 2024

A former physician’s personal Otter.ai AI scribe auto-joined a virtual hepatology rounds meeting via a stale calendar invite and emailed out a transcript containing protected health information. Reported to the Information and Privacy Commissioner of Ontario (Reported Breach HR24-00691). This is Canadian PHIPA, not HIPAA, but it is the cleanest documented case of an unapproved AI tool capturing real PHI through shadow-AI behavior.

Vector: An unapproved meeting transcriber auto-joining via a calendar integration that persisted after the physician offboarded.

Ongoing (documented 2024–2025)

Academic sources (USC Price School) document clinicians routinely pasting "anonymized" patient data into ChatGPT (often incompletely de-identified) and characterize it as "technically, a data breach" because OpenAI servers are not HIPAA-compliant. This is a widespread practice, not a single discrete named incident.

Vector: Clinicians pasting PHI into ChatGPT. For HIPAA, intent does not matter.

Reported Apr–May 2023

Bloomberg and The Register reported that Samsung banned employee use of generative AI after discovering staff had uploaded sensitive source code into ChatGPT; the internal policy warned of discipline up to termination. The frequently-repeated specific detail ("three separate leaks in 20 days," involving semiconductor source code, defect-detection algorithms, and a recorded meeting transcript) traces to a single Korean outlet (Economist Korea) and should be attributed as reported, not stated as established fact. Samsung itself confirmed a ban and "an incident."

Vector: Employees pasting source code and a meeting transcript into ChatGPT via the web browser. Legacy DLP is blind to browser copy/paste into AI.

January 2023

An Amazon lawyer warned employees over Slack not to share confidential information with ChatGPT, noting that outputs had "closely matched existing material." This is a documented hazard and internal warning, not a confirmed breach event.

Vector: Employees pasting code and confidential corporate information into ChatGPT.

February–May 2023

A wave of banks and large enterprises restricted or banned ChatGPT (and Copilot, in Apple’s case). The motivations were regulatory-risk and IP protection, largely pre-emptive rather than post-breach. JPMorgan reportedly could not determine how many employees were using ChatGPT or for what.

Vector: Employee use on unmanaged, personal accounts. The core problem was lack of visibility.

Disclosed Jan 29, 2025

Wiz research found DeepSeek had left a ClickHouse database publicly exposed and unauthenticated (ports 8123/9000), leaking over 1 million log lines including plaintext chat history, API keys and secrets, and backend metadata. Full database control was possible without authentication. DeepSeek secured it within hours of disclosure; no confirmed malicious third-party access.

Vector: A publicly accessible, unauthenticated ClickHouse database.

Access Jan 20–22, 2026; HHS notified June 5, 2026

A phishing-driven compromise of Xsolis, an AI utilization-management vendor, exposed the PHI of 1,396,519 individuals across 7 hospital systems: SSNs, insurance information, dates of birth, and treatment details. Filed on the HHS breach portal. Note: this is a phishing breach of an AI vendor, so the AI element is incidental to the attack vector.

Vector: Phishing compromise of a third-party AI vendor that is a HIPAA Business Associate.

March 20, 2023

A redis-py race condition in ChatGPT’s caching layer briefly exposed other users’ chat titles and first messages, and limited payment data (name, email, billing address, card type, and last four digits and expiry) for roughly 1.2% of ChatGPT Plus subscribers within a 9-hour window. No full card numbers were exposed; OpenAI notified affected users.

Vector: A caching bug creating cross-tenant leakage on a multi-tenant platform.

Disclosed Sept 2023 (token live from 2020)

Wiz found Microsoft’s own AI research team had exposed 38TB of private data via an overpermissive Azure SAS token in a public GitHub AI-training repo: disk backups of two employees’ workstations, secrets, private keys, passwords, and 30,000+ internal Teams messages. The token granted full control over the entire storage account. Microsoft revoked it and said no customer data was exposed.

Vector: An overpermissive Azure SAS token committed to a public GitHub repository.

Disclosed June 2025 · CVE-2025-32711 · CVSS 9.3

Aim Security (Aim Labs) researchers Pavan Reddy and Aditya Sanjay Gujral disclosed the first known zero-click prompt-injection data-exfiltration flaw in a production LLM. Microsoft’s advisory described it as "AI command injection in M365 Copilot allows an unauthorized attacker to disclose information over a network." Microsoft patched it server-side and found "no evidence" it was exploited maliciously in the wild.

Vector: Zero-click indirect prompt injection via a crafted email, exploiting RAG context inheritance, reaching anything in Copilot’s scope: emails, OneDrive, SharePoint, Teams.

Reported Jul 28, 2025 · disclosed Sept 25, 2025 · CVSS 9.4

Noma Labs demonstrated the same class of flaw in Salesforce Agentforce: an indirect prompt injection hidden in a Web-to-Lead form’s "Description" field, chained with a content-security-policy bypass via an expired allowlisted domain that researchers re-registered for about $5. Salesforce patched it via Trusted URL enforcement on Sept 8, 2025.

Vector: Prompt injection via an ingested lead form, plus a CSP bypass through a lapsed allowlisted domain, reaching CRM data: customer PII, pipeline, internal communications.

Aug 8–18, 2025 (disclosed late Aug 2025)

Google GTIG / Mandiant reported that the actor tracked as UNC6395 used stolen OAuth and refresh tokens from the Drift AI chat integration to reach Salesforce customer instances at "over 700 potentially impacted organizations." Attackers harvested Salesforce data and secrets: AWS keys, Snowflake credentials, and tokens for Slack, Google Workspace, S3, Azure, and OpenAI. Salesforce and Salesloft revoked tokens and pulled Drift from AppExchange; Cloudflare confirmed 104 API tokens compromised.

Vector: Stolen OAuth / refresh tokens from an AI chat integration, a non-human identity with broad SaaS-to-SaaS scope.

Ban Mar 2023 · €15M fine Dec 2024 · fine annulled Mar 18, 2026

Italy’s Garante temporarily banned ChatGPT in March 2023 (lifted April 2023 after remediation) and later fined OpenAI €15 million in December 2024 over data processing and training without a clear legal basis, and for not reporting the March 2023 breach to the regulator. Crucially, the Tribunale di Roma (Judgment no. 4153/2026, judge Damiana Colla) annulled the €15M fine on March 18, 2026, on jurisdictional grounds (Ireland’s DPC became lead authority Feb 15, 2024), not on the merits. Always cite the fine and its annulment together.

Vector: Data processing and model-training on personal data without a documented lawful basis under GDPR.

Ban Feb 2023 · €5M fine 2025

The Garante fined Replika’s maker €5 million for processing personal data without a legal basis and without age verification, and reaffirmed its ban in April 2025, citing risks to minors.

Vector: AI companion chatbot processing personal data, with no age verification.

Jan 30, 2025

The Garante imposed a definitive limitation on processing Italian users’ personal data by DeepSeek, citing inadequate GDPR answers and data stored in China; the app was pulled from Italian app stores.

Vector: A fast-adopted foreign AI tool with data residency in China and inadequate GDPR responses.

Dec 19, 2023

The FTC settled with Rite Aid over AI facial-recognition surveillance deployed without accuracy or bias controls, imposing a five-year ban on facial recognition for surveillance and ordering deletion of the data and models. It was the FTC’s first "algorithmic unfairness" action. Rite Aid disputed the allegations.

Vector: AI a client deploys (facial recognition) rather than AI a client uses.

Sept 11, 2025

The FTC issued 6(b) study orders to seven companies operating consumer AI companion chatbots, examining data handling and child safety. This is an inquiry, not an enforcement action, but it may inform future rules or cases.

Vector: Consumer AI companion chatbots and their handling of user and minors’ conversation data.

Filed Aug 15, 2025 · consolidated Oct 22, 2025

A class action alleges that Otter’s Notetaker / OtterPilot auto-joined Zoom, Meet, and Teams calls and recorded them without all-party consent, and that transcripts were used to train models, bringing ECPA, CFAA, and CIPA claims (plus alleged biometric voiceprints under Illinois BIPA). Otter filed a motion to dismiss in January 2026 denying interception. There has been no ruling on the merits; these are unproven allegations.

Vector: Alleged: a meeting transcriber auto-joining calls and recording without all-party consent.

Sanctions Jun 22, 2023

Two attorneys and their firm were sanctioned $5,000 under Rule 11 for submitting a brief containing six fabricated case citations produced by ChatGPT, and were ordered to notify the judges named in the fake opinions. The harm is output fabrication (hallucination), not data exposure.

Vector: ChatGPT used for legal research, producing hallucinated citations. Not a data leak.

Ruling Feb 14, 2024

The BC Civil Resolution Tribunal found Air Canada liable for negligent misrepresentation after its website customer-service chatbot gave a passenger wrong bereavement-fare information, awarding C$812.02. The tribunal rejected the argument that the chatbot was a "separate legal entity." The harm is misrepresentation, not data exposure.

Vector: A client-facing chatbot giving inaccurate information. Not a data leak.