Shadow AI Incidents: A Sourced List of Real AI Data Leaks

Reported breach Dec 17, 2024

A former physician’s personal Otter.ai AI scribe auto-joined a virtual hepatology rounds meeting via a stale calendar invite and emailed out a transcript containing protected health information. Reported to the Information and Privacy Commissioner of Ontario (Reported Breach HR24-00691). This is Canadian PHIPA, not HIPAA, but it is the cleanest documented case of an unapproved AI tool capturing real PHI through shadow-AI behavior.

Vector: An unapproved meeting transcriber auto-joining via a calendar integration that persisted after the physician offboarded.

Ongoing (documented 2024–2025)

Academic sources (USC Price School) document clinicians routinely pasting "anonymized" patient data into ChatGPT (often incompletely de-identified) and characterize it as "technically, a data breach" because OpenAI servers are not HIPAA-compliant. This is a widespread practice, not a single discrete named incident.

Vector: Clinicians pasting PHI into ChatGPT. For HIPAA, intent does not matter.

Reported Apr–May 2023

Bloomberg and The Register reported that Samsung banned employee use of generative AI after discovering staff had uploaded sensitive source code into ChatGPT; the internal policy warned of discipline up to termination. The frequently-repeated specific detail ("three separate leaks in 20 days," involving semiconductor source code, defect-detection algorithms, and a recorded meeting transcript) traces to a single Korean outlet (Economist Korea) and should be attributed as reported, not stated as established fact. Samsung itself confirmed a ban and "an incident."

Vector: Employees pasting source code and a meeting transcript into ChatGPT via the web browser. Legacy DLP is blind to browser copy/paste into AI.

January 2023

An Amazon lawyer warned employees over Slack not to share confidential information with ChatGPT, noting that outputs had "closely matched existing material." This is a documented hazard and internal warning, not a confirmed breach event.

Vector: Employees pasting code and confidential corporate information into ChatGPT.

February–May 2023

A wave of banks and large enterprises restricted or banned ChatGPT (and Copilot, in Apple’s case). The motivations were regulatory-risk and IP protection, largely pre-emptive rather than post-breach. JPMorgan reportedly could not determine how many employees were using ChatGPT or for what.

Vector: Employee use on unmanaged, personal accounts. The core problem was lack of visibility.

Disclosed Jan 29, 2025

Wiz research found DeepSeek had left a ClickHouse database publicly exposed and unauthenticated (ports 8123/9000), leaking over 1 million log lines including plaintext chat history, API keys and secrets, and backend metadata. Full database control was possible without authentication. DeepSeek secured it within hours of disclosure; no confirmed malicious third-party access.

Vector: A publicly accessible, unauthenticated ClickHouse database.

Access Jan 20–22, 2026; HHS notified June 5, 2026

A phishing-driven compromise of Xsolis, an AI utilization-management vendor, exposed the PHI of 1,396,519 individuals across 7 hospital systems: SSNs, insurance information, dates of birth, and treatment details. Filed on the HHS breach portal. Note: this is a phishing breach of an AI vendor, so the AI element is incidental to the attack vector.

Vector: Phishing compromise of a third-party AI vendor that is a HIPAA Business Associate.

March 20, 2023

A redis-py race condition in ChatGPT’s caching layer briefly exposed other users’ chat titles and first messages, and limited payment data (name, email, billing address, card type, and last four digits and expiry) for roughly 1.2% of ChatGPT Plus subscribers within a 9-hour window. No full card numbers were exposed; OpenAI notified affected users.

Vector: A caching bug creating cross-tenant leakage on a multi-tenant platform.

Disclosed Sept 2023 (token live from 2020)

Wiz found Microsoft’s own AI research team had exposed 38TB of private data via an overpermissive Azure SAS token in a public GitHub AI-training repo: disk backups of two employees’ workstations, secrets, private keys, passwords, and 30,000+ internal Teams messages. The token granted full control over the entire storage account. Microsoft revoked it and said no customer data was exposed.

Vector: An overpermissive Azure SAS token committed to a public GitHub repository.

Disclosed July 16, 2026

Hugging Face disclosed that an autonomous AI-driven agent breached its production infrastructure over a weekend. In its own words, "a malicious dataset abused two code-execution paths in our dataset processing (a remote-code dataset loader and a template-injection in a dataset configuration)" to run code on a processing worker, after which the attacker harvested service credentials and moved laterally into internal clusters, running thousands of individual actions across a swarm of short-lived sandboxes. Hugging Face reported "unauthorized access to a limited set of internal datasets and to several credentials used by our services," but said it found "no evidence of tampering with public, user-facing models, datasets, or Spaces" and verified its software supply chain clean. It added it was "still completing our assessment of whether any partner or customer data was affected."

Vector: Code execution in the AI platform’s own dataset-processing pipeline, escalated by an autonomous agent framework into credential theft and lateral movement. One of the first publicly reported cases of an agentic attacker breaching a major AI platform.

Campaign Jul 21 to 22, 2026 · disclosed Jul 22, 2026

Huntress disclosed a malvertising campaign it named FakeAgent. In its words, "Between July 21 and July 22, at least 29 organizations fell victim to a malvertising campaign that led them to a malicious Claude Artifact." Employees searching Bing for the Claude desktop app clicked a sponsored result that pointed at the genuine claude.ai domain: "Instead of sending the user to a normal page, the ad pointed to the public artifact hosted on Claude.ai." Huntress reported the artifact link "had 7,100 page views" before Anthropic removed it. Visitors were redirected to attacker infrastructure serving a fake ClaudeDesktop.exe, a legitimate JetBrains Chromium binary that sideloaded a malicious libcef.dll to run the SectopRAT infostealer, whose plaintext strings "include references to browser logins, cookies, autofills, credit cards, chromium key theft, FTP, Discord, messaging clients, etc." Separately, Check Point Research’s Q2 2026 Brand Phishing Report, published July 23, 2026, put ChatGPT in the top ten most impersonated brands for the first time, at 1.1% of recorded brand-impersonation phishing.

Vector: Employees sourcing an AI desktop app themselves. A paid search ad resolving to a real claude.ai URL, then a redirect to a fake installer delivering a credential-stealing remote access trojan. Nothing in the address bar looked wrong.

Reported Jan 28, 2026 · disclosed Aug 4, 2026

Security researcher bobdahacker published that tl;dv, an AI notetaker that joins and transcribes Zoom, Google Meet, and Microsoft Teams calls, had left its Firestore "meetings" collection with no tenant isolation. In the researcher’s words: "Any authenticated tl;dv user can query every meeting across every account on the platform." The researcher counted 181,874 meeting records covering 84,312 unique users across 35,003 email domains. Each record carried "the creator’s email address, the conference ID (which is a joinable Google Meet or Teams room), the provider, the recording status, and timestamps," with roughly 1,000 meetings showing status: recording at any given moment. The researcher then demonstrated the consequence: "Grabbed a conference ID from Firestore and joined a live Google Meet belonging to the Malaysian Ministry of Education." Separately, scraping 27,334 meeting IDs surfaced more than 1,000 publicly shared meetings and 715 invitee email addresses across 228 domains. The researcher documented follow-ups through July 22, 2026 with the flaw still open, then published on August 4. tl;dv acknowledged "a vulnerability related to access to specific conference metadata" and said corrective measures were taken, disputing that anything sat unfixed for six months and describing "two different attack vectors" instead. Dark Reading and Gigazine reported the same findings. What was reachable was meeting metadata and joinable conference IDs, not transcripts by default; the "more than two million users" figure that appears in secondary coverage is the vendor’s own marketing number and is not the researcher’s count.

Vector: A missing tenant-isolation rule on one collection in an AI notetaker’s own database, reachable with any ordinary user’s Firebase token, turning the vendor into a cross-tenant index of who was meeting whom, and handing out conference IDs for calls that were live at that moment.

Disclosed June 2025 · CVE-2025-32711 · CVSS 9.3

Aim Security (Aim Labs) researchers Pavan Reddy and Aditya Sanjay Gujral disclosed the first known zero-click prompt-injection data-exfiltration flaw in a production LLM. Microsoft’s advisory described it as "AI command injection in M365 Copilot allows an unauthorized attacker to disclose information over a network." Microsoft patched it server-side and found "no evidence" it was exploited maliciously in the wild.

Vector: Zero-click indirect prompt injection via a crafted email, exploiting RAG context inheritance, reaching anything in Copilot’s scope: emails, OneDrive, SharePoint, Teams.

Reported Jul 28, 2025 · disclosed Sept 25, 2025 · CVSS 9.4

Noma Labs demonstrated the same class of flaw in Salesforce Agentforce: an indirect prompt injection hidden in a Web-to-Lead form’s "Description" field, chained with a content-security-policy bypass via an expired allowlisted domain that researchers re-registered for about $5. Salesforce patched it via Trusted URL enforcement on Sept 8, 2025.

Vector: Prompt injection via an ingested lead form, plus a CSP bypass through a lapsed allowlisted domain, reaching CRM data: customer PII, pipeline, internal communications.

Aug 8–18, 2025 (disclosed late Aug 2025)

Google GTIG / Mandiant reported that the actor tracked as UNC6395 used stolen OAuth and refresh tokens from the Drift AI chat integration to reach Salesforce customer instances at "over 700 potentially impacted organizations." Attackers harvested Salesforce data and secrets: AWS keys, Snowflake credentials, and tokens for Slack, Google Workspace, S3, Azure, and OpenAI. Salesforce and Salesloft revoked tokens and pulled Drift from AppExchange; Cloudflare confirmed 104 API tokens compromised.

Vector: Stolen OAuth / refresh tokens from an AI chat integration, a non-human identity with broad SaaS-to-SaaS scope.

Advisory Jul 13, 2026 · exploited in the wild Jul 18, 2026 · CVE-2026-6875 · CVSS 9.5

Searchlight Cyber researchers disclosed a critical, unauthenticated sandbox-escape flaw in the ServiceNow AI Platform, the workflow platform ServiceNow says runs at roughly 85% of the Fortune 500. Its GlideRecord query API accepted a "javascript:" prefix in addQuery() calls, so an attacker could execute code with no authentication and, per the researchers, achieve full compromise of an instance: read data in any table, create administrator accounts, and run commands on connected MID Server proxies. ServiceNow patched hosted instances starting in April and released the self-hosted fix on July 13, 2026 (advisory KB3137947). By Friday, July 18, Defused researchers observed active in-the-wild exploitation, which sets this apart from EchoLeak and ForcedLeak, neither of which was ever exploited. ServiceNow said it has "not observed evidence that this activity is related to instances that ServiceNow hosts," and no data theft has been confirmed yet.

Vector: An unauthenticated pre-auth remote-code-execution sandbox escape in an enterprise AI platform, reachable before login and now being exploited against unpatched self-hosted instances.

Reported Jun 4, 2026 · fixed Jun 8, 2026 · disclosed Jul 23, 2026

Zenity Labs disclosed a cross-site request forgery flaw in ChatGPT’s Agent Builder that it named AgentForger. The Builder accepted an initialization state through URL parameters, and the value of initial_assistant_prompt was not merely placed into the prompt box but automatically submitted and executed, so one click on an attacker-crafted ChatGPT link could build, configure, and publish an agent inside the victim’s workspace with approval gates disabled. Zenity’s description: "it forges an entire autonomous agent, attacker-controlled and operating inside the organization’s trust boundary." Zenity co-founder and CTO Michael Bargury put it this way: "This isn’t a forged request, it’s a forged insider. With one click, an attacker gets a fully autonomous agent inside your company that has your people’s identity and access, with the guardrails off." The forged agent inherited whatever connectors the employee had already authorized, including Outlook, Gmail, Slack, Google Drive, SharePoint, Teams, and calendars, and could keep running on a schedule and take commands by email. Zenity reported it through Bugcrowd on June 4, 2026; OpenAI triaged it within a day and fixed it on June 8, 2026 by removing the vulnerable URL parameter handler. There is no public evidence of exploitation in the wild.

Vector: A single phishing link to a legitimate ChatGPT URL, abusing an over-permissive URL parameter in Agent Builder to stand up a persistent, attacker-controlled agent carrying the victim’s identity and already-granted OAuth app access.

Disclosed Jul 23, 2026 · chains CVE-2026-46331

Researchers at Accomplish AI published a sandbox escape in the local execution mode of Claude Cowork, Anthropic’s agentic desktop product, which they named SharedRoot. Their finding, verbatim: "Untrusted content in a Claude Cowork session can escape the VM it’s sandboxed in and read and write files anywhere on your Mac." The chain used CVE-2026-46331, a public Linux kernel flaw in the act_pedit module known as pedit COW, to gain root inside the agent’s Linux virtual machine, then walked out through a writable mount that exposed the host filesystem. What became reachable on the Mac, in the researchers’ words: "SSH keys, cloud credentials, anything the user’s account can touch." Researcher Oren Yomtov described the demonstration: "We connected a folder to a fresh Claude Cowork session, sent one short message, and watched the agent escape the sandbox." Accomplish AI said the issue "has been reported to Anthropic and was closed as ‘Informative’" without a fix, and that "Cowork now uses cloud execution by default, and this local escape path does not appear to apply there." Sessions still configured to run locally remain the exposure. Anthropic has not published its own account.

Vector: A local AI agent running against a shared host filesystem mount. Content handed to the agent escalates privileges inside its virtual machine and then reads and writes across the user’s Mac, far outside the folder the user connected, with no permission prompt.

Published Aug 6, 2026 · title corrected Aug 11, 2026 · CVSS 9.3

The small-tenant version first: a 60-seat client with Microsoft 365 Copilot licences can have an autonomous agent running multi-step work across its mail, files, and meetings because one admin flipped a switch, and that agent just took a critical authorization CVE. Microsoft published CVE-2026-59118 against Copilot Cowork, the agentic product that reached general availability on June 16, 2026 and which, in Microsoft’s words, "requires the Microsoft 365 Copilot User Subscription License (USL)" with users then "billed for Cowork on a usage-based basis." The advisory reads, verbatim: "Improper authorization in Copilot Cowork allows an unauthorized attacker to elevate privileges over a network." CWE-285. The CVSS 3.1 base score is 9.3, vector AV:N/AC:L/PR:N/UI:R/S:C/C:H/I:H/A:N, so no privileges are required, one user interaction is, and the scope is changed. Microsoft recorded Publicly Disclosed: No and Exploited: No, set Customer Action Required to No, and stated: "This vulnerability has already been fully mitigated by Microsoft. There is no action for users of this service to take. The purpose of this CVE is to provide further transparency." Credit went to Ilan Kalendarov, Elad Beber, and Ben Zamir of Cymulate.

Vector: An authorization flaw in an AI agent Microsoft built to plan and execute multi-step tasks inside a tenant, grounded in the client’s own mail, meetings, files, and connected line-of-business apps. There was nothing to patch on an endpoint: the exposure lived in the service, and the only lever a customer ever held was whether Cowork was switched on and for whom.

Reported to Microsoft Dec 2025 · patched Aug 18, 2026 · CVSS 3.1 8.8

Varonis Threat Labs disclosed CoSnitch, a three-flaw chain in Microsoft Copilot Personal, the free consumer assistant built into Windows 11 and reachable at copilot.microsoft.com. This is a different product from the enterprise Microsoft 365 Copilot behind EchoLeak and Copilot Cowork above, and the research does not state the same behavior affected that enterprise product. In Varonis’s words, CoSnitch is "a one-click flaw that silently exfiltrates data." The chain combined three weaknesses: an undocumented URL parameter, autorun=1, paired with the existing q parameter, that triggered automatic prompt execution the moment a crafted link loaded, with no click inside the page required; Copilot then used its own already-granted access to the victim’s connected Gmail, Google Drive, and Google Calendar to pull message bodies, file metadata, and calendar details, and shipped them out through Copilot’s own URL-fetch capability to an attacker-controlled server; and a persistent memory-poisoning step, in which a crafted webpage that Copilot merely summarized could implant an instruction into the user’s permanent memory store, which Varonis says "survives password changes, session revocation, and device re-enrollment, persisting forever" until manually removed. On detectability, Varonis wrote: "From the network layer, this is a standard outbound HTTPS GET request," indistinguishable from any legitimate Copilot fetch. Researchers found the hidden parameter by repeatedly questioning Copilot itself rather than reverse-engineering its code, a technique Varonis summarized as: "Copilot wasn’t breached; it was played." Varonis reported CoSnitch to Microsoft in December 2025; Microsoft shipped the fix on August 18, 2026 as CVE-2026-24301, describing it in its own advisory as "improper neutralization of special elements used in a command ('command injection') in Microsoft Copilot allows an unauthorized attacker to disclose information over a network," rated 8.8 under CVSS 3.1. Varonis states it "has seen no evidence that the attack has been exploited in the wild."

Vector: A single crafted link to the free consumer Copilot at copilot.microsoft.com, the assistant on the Windows 11 taskbar that any employee can use with a personal Microsoft account and no enterprise licence, reaching whatever personal Gmail, Drive, or Calendar that employee has connected to it.

Ban Mar 2023 · €15M fine Dec 2024 · fine annulled Mar 18, 2026

Italy’s Garante temporarily banned ChatGPT in March 2023 (lifted April 2023 after remediation) and later fined OpenAI €15 million in December 2024 over data processing and training without a clear legal basis, and for not reporting the March 2023 breach to the regulator. Crucially, the Tribunale di Roma (Judgment no. 4153/2026, judge Damiana Colla) annulled the €15M fine on March 18, 2026, on jurisdictional grounds (Ireland’s DPC became lead authority Feb 15, 2024), not on the merits. Always cite the fine and its annulment together.

Vector: Data processing and model-training on personal data without a documented lawful basis under GDPR.

Ban Feb 2023 · €5M fine 2025

The Garante fined Replika’s maker €5 million for processing personal data without a legal basis and without age verification, and reaffirmed its ban in April 2025, citing risks to minors.

Vector: AI companion chatbot processing personal data, with no age verification.

Jan 30, 2025

The Garante imposed a definitive limitation on processing Italian users’ personal data by DeepSeek, citing inadequate GDPR answers and data stored in China; the app was pulled from Italian app stores.

Vector: A fast-adopted foreign AI tool with data residency in China and inadequate GDPR responses.

Dec 19, 2023

The FTC settled with Rite Aid over AI facial-recognition surveillance deployed without accuracy or bias controls, imposing a five-year ban on facial recognition for surveillance and ordering deletion of the data and models. It was the FTC’s first "algorithmic unfairness" action. Rite Aid disputed the allegations.

Vector: AI a client deploys (facial recognition) rather than AI a client uses.

Sept 11, 2025

The FTC issued 6(b) study orders to seven companies operating consumer AI companion chatbots, examining data handling and child safety. This is an inquiry, not an enforcement action, but it may inform future rules or cases.

Vector: Consumer AI companion chatbots and their handling of user and minors’ conversation data.

Filed Aug 15, 2025 · consolidated Oct 22, 2025

A class action alleges that Otter’s Notetaker / OtterPilot auto-joined Zoom, Meet, and Teams calls and recorded them without all-party consent, and that transcripts were used to train models, bringing ECPA, CFAA, and CIPA claims (plus alleged biometric voiceprints under Illinois BIPA). Otter filed a motion to dismiss in January 2026 denying interception. The court ruled on that motion on August 13, 2026, granting it in part and letting the core wiretap and biometric claims proceed (see the next entry). There has still been no ruling on the merits; these remain unproven allegations.

Vector: Alleged: a meeting transcriber auto-joining calls and recording without all-party consent.

Order Aug 13, 2026 · N.D. Cal. No. 5:25-cv-06911-EKL · Dkt. 68

Judge Eumi K. Lee granted Otter.ai’s motion to dismiss in part and denied it in part, and the claims that carry the exposure survived. The order opens: "This putative class action alleges that Defendant Otter.ai Inc. (‘Otter’) surreptitiously eavesdrops and records conversations held over virtual meeting platforms like Zoom and Microsoft Teams without obtaining consent from all meeting participants." On the federal Wiretap Act count the court held that plaintiffs "plausibly allege a contemporaneous acquisition of their communications because Otter allegedly records and processes meetings ‘in real time’ as they occur," and rejected Otter’s party-exception defence because "committing a tort and seeking a profit are not mutually exclusive." Surviving: the ECPA count (Count 1), the CIPA count (Count 8, on Cal. Penal Code § 631(a)), and both Illinois BIPA voiceprint counts (15 and 16, under 740 ILCS 14/15(a) and 15(b)). Dismissed with leave to amend: the CFAA count, the California CDAFA count, intrusion upon seclusion as to three plaintiffs, the California constitutional privacy count as to two, and the Washington Privacy Act count. Otter’s Article III standing challenge failed outright: "the motion to dismiss Plaintiffs’ claims for lack of standing is denied." Plaintiffs had 14 days to amend and Otter 21 days after that to respond. This is a pleading-stage ruling on what was plausibly alleged, not a finding that Otter did anything unlawful.

Vector: Alleged: an AI notetaker joining Zoom, Meet, and Teams calls as a "silent participant" and recording, transcribing, and retaining what was said, which the complaint says included "sensitive medical, financial, and professional discussions," plus voiceprints retained to train speech-recognition and machine-learning models.

Filed Jul 30, 2026 · N.D. Cal. No. 3:26-cv-07926-EMC

Tarra Chamberlain, a Florida resident who was never a Granola user, filed a putative class action over Granola’s bot-free AI notetaker. The complaint opens: "This case concerns spyware. Granola designed and created an AI notetaking product that surreptitiously intercepts, records, and interprets the communications of every participant in virtual meetings without their knowledge or consent." It alleges Granola captures meetings through the computer’s "system audio and microphone" instead of joining as a visible bot, so it "can transcribe any call where the audio plays through your computer," and it quotes Granola’s own marketing that "[o]ther people in the room won’t know" it is there is the "distinction that matters most," alongside "No bot joins the call. No notification appears." It also quotes Granola’s privacy policy that data "incorporated into AI or analytics models or other databases will not be removed from those models and datasets, as removal may not be technically feasible without complete model retraining or database reconstruction." Claims include the federal ECPA, CIPA § 631 and § 632, and the California Comprehensive Computer Data Access and Fraud Act (Cal. Penal Code § 502), with statutory damages sought under Cal. Penal Code § 637.2(a) of the greater of five thousand dollars per violation or treble damages. Summons was returned executed on August 5, 2026, with a response due August 26. Holland & Knight reports a second late-July filing on the same theory, Thompson v. SoundHound AI, Inc., 26-cv-202181 (Alameda County Superior Court), over an AI phone-ordering system deployed at restaurants; that case is reported by one outlet and its exact filing date is not public. These are unproven allegations.

Vector: Alleged: a desktop AI notetaker that captures the computer’s system audio and microphone with no bot in the meeting and no notification, so no attendee, no meeting host, and no inventory that looks for meeting bots can see it running.

Sanctions Jun 22, 2023

Two attorneys and their firm were sanctioned $5,000 under Rule 11 for submitting a brief containing six fabricated case citations produced by ChatGPT, and were ordered to notify the judges named in the fake opinions. The harm is output fabrication (hallucination), not data exposure.

Vector: ChatGPT used for legal research, producing hallucinated citations. Not a data leak.

Ruling Feb 14, 2024

The BC Civil Resolution Tribunal found Air Canada liable for negligent misrepresentation after its website customer-service chatbot gave a passenger wrong bereavement-fare information, awarding C$812.02. The tribunal rejected the argument that the chatbot was a "separate legal entity." The harm is misrepresentation, not data exposure.

Vector: A client-facing chatbot giving inaccurate information. Not a data leak.