Calendar Icon White
April 27, 2026
Clock Icon
6
 min read

ChatGPT Security Risks in Enterprise: 2026 Guide to Data Leaks, Breaches & Prevention

Explore ChatGPT security risks, from data breaches to malware, and learn the best practices to protect your organisation's sensitive data.

ChatGPT Security Risks in Enterprise: 2026 Guide to Data Leaks, Breaches & Prevention
ChatGPT
Perplexity
Grok
Google AI
Claude
Summarize and analyze this article with:

TL;DR

  • The real risk isn't the model — it's your employees. Pasting PII, source code, customer records, and credentials into ChatGPT is the #1 cause of ChatGPT data leaks in enterprise.
  • OpenAI protects the pipes, not the content. TLS 1.2+, AES-256, SOC 2 Type II, and Zero Data Retention (ZDR) on Enterprise/API tiers are real — but none of them stop a user from pasting PHI into a prompt.
  • ChatGPT Free, Plus, and Team train on your data by default. Only Enterprise and the API come with ZDR by default. Most "shadow AI" incidents happen on personal Plus accounts your IT team never authorized.
  • Real breaches keep happening: Samsung (source code, 2023), OpenAI's Redis bug (chat titles + billing data exposed, 2023), 225,000+ ChatGPT credentials on the dark web (Group-IB, 2023), prompt injection exfiltrating ChatGPT memory (2024).
  • The fix is boring but effective: detect and redact sensitive data before it leaves the browser. Strac's ChatGPT DLP does exactly this — block, warn, or audit in real time, with no proxy and no TLS break.

ChatGPT Security Risks in Enterprise: 2026 Guide to Data Leaks, Breaches & Prevention

Strac protects sensitive data across Claude, ChatGPT, and 50+ AI tools — detecting PII, PCI, and PHI in real time before prompts leave the browser
Strac's browser extension detects SSNs, credit card numbers, and medical record numbers in ChatGPT, Claude, and 50+ other AI tools — and blocks before submission

What ChatGPT Protects (and What It Doesn't)

ChatGPT encrypts data in transit (TLS 1.2+) and at rest (AES-256). OpenAI is SOC 2 Type II certified, offers ChatGPT Enterprise with SAML SSO, SCIM, audit logs, and customer-managed encryption keys. ChatGPT Enterprise and the API default to Zero Data Retention on prompts — OpenAI does not train on your data, and conversations aren't retained beyond 30 days (often less).

All of that is real. None of it addresses the actual security question enterprises face:

What did the employee paste into the prompt?

Encryption protects data moving between your browser and OpenAI's servers. It does not protect data moving into the prompt in the first place. That's the gap, and it's where every significant ChatGPT security incident has happened.

✨ The 7 Real ChatGPT Security Risks in Enterprise

Stack-ranked by how often we see them in customer environments — not by theoretical severity.

1. Employees pasting sensitive data into prompts

The single largest ChatGPT security risk. Cyberhaven's 2023 research found that at organizations averaging 100,000 employees, confidential data was entered into ChatGPT ~200 times per week. That data included source code, customer records, financial forecasts, and internal strategy documents.

Once a prompt leaves the browser, you've lost control of it. Even on ChatGPT Enterprise with ZDR, that data is now in a transcript your employees can screenshot, forward, or re-paste into unsanctioned tools.

Real-time detection the moment a user pastes or types — no data leaves the browser until policy says it can

2. Personal ChatGPT accounts used for work ("shadow AI")

Your company may have purchased ChatGPT Enterprise. Your employees may not be using it. They'll sign up for personal ChatGPT Plus accounts with their personal email, paste work data, and bypass every control your IT team built. On Free and Plus tiers, OpenAI trains on that data by default.

Enterprise buyers consistently underestimate this. The ratio of sanctioned-to-shadow ChatGPT usage in most organizations is roughly 1:3.

3. Credentials, API keys, and secrets in prompts

Developers debugging code paste everything: config files, environment variables, private keys, database connection strings. A 2023 Group-IB analysis found 225,000+ stolen ChatGPT account credentials circulating on dark web marketplaces, most harvested by infostealer malware. Those accounts contained conversation histories loaded with proprietary information — including credentials.

4. Prompt injection attacks on integrated ChatGPT

As soon as ChatGPT reads external content — a URL, a PDF, a shared document, a Slack message — that content can contain hidden instructions. Johann Rehberger's disclosed vulnerability in ChatGPT's memory feature (2024) demonstrated exfiltration of conversation data via prompt injection. The attack surface expands every time OpenAI ships a new feature (Search, Code Interpreter, Connectors, Agents).

5. Third-party plugins, GPTs, and connectors

OpenAI reviews plugins and custom GPTs, but data sent to a third-party plugin follows that plugin's privacy policy — not OpenAI's. Custom GPTs have repeatedly leaked system prompts, uploaded knowledge files, and attached documents through prompt injection. Enterprise connectors to SharePoint, Google Drive, and GitHub extend the risk surface into your most sensitive systems.

6. Regulated data going to a non-BAA model

Unless you have a Business Associate Agreement (BAA) with OpenAI — available on ChatGPT Enterprise — sending PHI to ChatGPT is a HIPAA violation. PCI DSS data has similar constraints. Most ChatGPT Free, Plus, and Team users at healthcare and fintech companies don't realize their prompts are non-compliant until an audit surfaces it.

7. Exfiltration via ChatGPT Search and browsing features

When ChatGPT fetches a URL on behalf of the user, a malicious page can inject instructions telling ChatGPT to exfiltrate conversation history to an attacker-controlled domain. Multiple variants of this attack have been documented in 2024 and 2025.

Cut from most "ChatGPT security" articles (and rightly so for enterprise buyers): sponge sampling, model inversion, training data reconstruction, data poisoning. These are real research topics but essentially irrelevant to an enterprise CISO deciding whether to approve ChatGPT usage. The risks above are what actually cause incidents.

✨ Recent ChatGPT Data Leaks and Security Breaches

Samsung source code leak (2023)

Within 20 days of Samsung allowing ChatGPT internally, three separate engineers pasted confidential information into the chatbot — including semiconductor source code and internal meeting recordings transcribed for summarization. Samsung banned ChatGPT enterprise-wide shortly after. The Samsung incident remains the single most-cited example of why ChatGPT usage needs detection controls before the prompt, not just at the network layer.

Block or Warn — Strac's DLP modal gives users clear policy feedback with granular per-data-type enforcement
Block the submission outright, warn and let the user decide, or audit silently in the background — the control Samsung didn't have

OpenAI's Redis bug exposing chat titles and billing data (March 2023)

A bug in the Redis open-source client caused ChatGPT to show other users' chat titles to random users, and exposed payment-related data (first and last names, email addresses, payment addresses, last four digits of credit card numbers, and card expiration dates) of roughly 1.2% of ChatGPT Plus subscribers active during a nine-hour window. OpenAI disclosed and patched the issue; the incident prompted Italy's data protection authority to temporarily ban ChatGPT nationwide.

225,000+ ChatGPT credentials on the dark web (Group-IB, 2023)

Group-IB's threat intelligence team identified stolen ChatGPT credentials in logs from Raccoon, Vidar, and RedLine infostealer malware, sold on dark web marketplaces throughout 2023. These accounts gave attackers full access to chat histories — essentially, employee confessions of what their companies were working on.

ChatGPT memory feature exfiltration (2024)

Security researcher Johann Rehberger disclosed that ChatGPT's memory feature could be abused through prompt injection — an attacker could plant instructions in documents or web pages ChatGPT read, which would then write persistent malicious entries into a user's long-term memory. Those entries could include instructions to silently exfiltrate future conversations.

Italian DPA action and EU regulatory scrutiny (ongoing)

Italy's Garante has repeatedly opened investigations into ChatGPT for GDPR violations — including insufficient legal basis for training data, inadequate age verification, and improper handling of European user data. Fines have reached €15 million (December 2024). Other EU DPAs (France, Germany, Spain) have opened parallel investigations.

The pattern: nearly every disclosed ChatGPT security incident traces back to data that never should have entered a prompt in the first place — or to an integration that exposed conversations to third parties. Encryption and SOC 2 certifications don't help with either.

ChatGPT Security by User Tier

Enterprise buyers often assume "we're on ChatGPT, we're fine." The tier matters enormously.

Tier
ZDR default
Trained on your data?
SSO/SCIM
Audit logs
BAA available
Suitable for regulated data?
Free
No
Yes
No
No
No
Never
Plus ($20/mo personal)
No
Yes (unless opted out)
No
No
No
Never
Team
No
No
Limited
Limited
No
Not for PHI/PCI
Enterprise
Yes
No
Yes
Yes
Yes
Yes, with BAA
API (with data controls)
Yes
No
Via your app
Via your app
Yes
Yes

The overwhelming majority of ChatGPT-related data leaks in enterprise happen on Free and Plus tiers — accounts your IT team didn't provision and can't govern. Buying ChatGPT Enterprise does not solve this on its own. You also need a way to detect when employees are using personal accounts on corporate devices.

✨ ChatGPT Data Security in Third-Party Integrations

ChatGPT Enterprise connectors to Google Drive, SharePoint, GitHub, and Slack are powerful — and they multiply the attack surface. A single misconfigured connector scope can expose every file a user has read access to, not just what they meant to share with ChatGPT. Meanwhile, building ChatGPT into your own products via the API exposes customer data to OpenAI's infrastructure (even under ZDR, metadata flows through).

Common integration risks:

  • Connector over-scoping. Granting ChatGPT read access to an entire SharePoint tenant when the user only needed one folder.
  • Prompt injection via shared documents. Any document ChatGPT can read can contain embedded instructions.
  • Unsanctioned Zapier/Make.com pipes that forward Slack messages, Zendesk tickets, or Jira comments to ChatGPT for "AI summarization" without data security controls in the middle.
  • OAuth token leakage. Connectors store long-lived tokens that, if breached, give persistent access to source systems.
Strac redacts sensitive data in Slack in real time — before it ever reaches a ChatGPT connector, Zapier pipe, or downstream AI integration

Mitigation checklist for integrated ChatGPT:

  • Audit every OAuth grant and connector scope quarterly
  • Treat any document ChatGPT can read as a potential injection vector
  • Place detection and redaction between your SaaS tools (Slack, Zendesk, Jira) and ChatGPT — not just at the ChatGPT endpoint
  • Require BAAs and DPAs on every integration touching regulated data

ChatGPT Enterprise Security: What OpenAI Gives vs. What's Still Your Job

What ChatGPT Enterprise covers out of the box:

  • Encryption in transit and at rest
  • SAML SSO and SCIM provisioning
  • Admin console with user/workspace management
  • Audit logs (API access)
  • Domain verification
  • SOC 2 Type II, GDPR, BAA available
  • Zero Data Retention on prompts; no training on your data

What ChatGPT Enterprise does NOT cover:

  • Content inspection of prompts. OpenAI won't stop an employee from pasting a customer's SSN.
  • Shadow ChatGPT usage. Employees using personal accounts on work devices are invisible to the Enterprise admin console.
  • Downstream exfiltration. Once a response lands in the user's browser, where it goes next (email, Slack, another AI tool) is not OpenAI's problem.
  • Cross-tool policy enforcement. A policy forbidding PHI in ChatGPT doesn't apply when the same data moves through Slack, Jira, or SharePoint.
  • Regulated-industry redaction. OpenAI signs BAAs but doesn't redact PHI for you.

This gap is what Strac is built to close.

ChatGPT Security Concerns for Regulated Industries

Healthcare (HIPAA)

PHI in a ChatGPT prompt without a signed BAA is a HIPAA violation — full stop. Even with a BAA (Enterprise only), HIPAA's minimum-necessary principle requires that only the minimum PHI needed be disclosed. That means de-identification or redaction before the prompt is sent, not after.

Financial services (PCI DSS, GLBA, SOX)

PCI DSS Requirement 4 prohibits unencrypted transmission of cardholder data over open, public networks — but also requires strict controls on where cardholder data can live. Sending a PAN into ChatGPT, even with ZDR, creates audit exposure. GLBA and SOX similarly require documented controls over customer and financial data handling.

EU GDPR

Article 32 requires appropriate technical and organizational measures. Pasting EU personal data into any LLM without DPIA, explicit lawful basis, and data minimization controls almost always fails this bar. Italy's ongoing ChatGPT enforcement is the warning shot.

Education (FERPA)

Student records in ChatGPT without consent violate FERPA. Most schools have no DLP controls around ChatGPT usage by faculty — an audit risk that's grown rapidly since 2023.

✨ 🎥 How to Prevent ChatGPT Data Leaks with Strac

Strac closes the content-inspection gap with four layers, each deployable independently or together.

1. ChatGPT DLP in the browser

A Chrome/Edge/Firefox extension that inspects every ChatGPT interaction (chat.openai.com, chatgpt.com, enterprise tenants, custom GPTs) in real time — before data leaves the browser. Three enforcement modes:

  • Block. Sensitive data is refused outright. User sees a clear message explaining which policy triggered.
  • Warn. User sees a warning with the redacted preview and chooses whether to proceed.
  • Audit (silent). Logs everything for your security team without interrupting users — ideal for baseline discovery before enforcement.

Detection covers 100+ sensitive types out of the box: SSNs, driver's licenses, passport numbers, credit cards, bank accounts, medical record numbers, API keys, AWS credentials, OAuth tokens, private keys, and custom regex patterns for your internal data (employee IDs, project codenames, case numbers).

Deploy the Strac extension through Chrome Enterprise or Edge Group Policy in under 10 minutes — no proxy, no TLS break
No proxy, no TLS interception, no network changes. Install the extension, connect to the Strac console, set policy, done

2. Shadow AI discovery on the endpoint

Strac's endpoint agent discovers which AI tools your employees are actually using — including personal ChatGPT Plus, personal Claude, Microsoft Copilot, Perplexity, and locally-run models. Enterprise email enforcement nudges users off personal accounts when a corporate alternative exists.

3. Integration-level data security

Redact sensitive data in the SaaS tools that feed ChatGPT connectors — Slack, Zendesk, Jira, Salesforce, Google Drive, SharePoint, Box. If ChatGPT never sees the raw PHI in a Slack message, it can't accidentally include it in a summary.

4. MCP DLP for agentic workflows

As AI agents start calling tools on employees' behalf via MCP (Model Context Protocol), Strac inspects and redacts data at the MCP server boundary — blocking sensitive data from being sent to agents, or redacting it inline before the agent sees it.

Strac inspects and redacts sensitive data at the MCP server boundary — before the agent ever sees it
MCP DLP sits between your agent and its tools, redacting PII/PHI/credentials before they flow into the model's context
Strac's agentless data security across every SaaS tool, cloud, endpoint, and AI surface your employees touch
Strac's coverage spans 50+ integrations — agentless, deploys in under 10 minutes, no proxy, no TLS break

Deployment is agentless on the SaaS side and takes roughly 10 minutes. The browser extension rolls out via Chrome Enterprise, Edge for Business, or Jamf in a standard MDM flow.

ChatGPT Security Checklist

Use this as a monthly audit for your organization:

  • [ ] Have you inventoried every ChatGPT tier in use (Free, Plus, Team, Enterprise, API)?
  • [ ] Have you blocked or replaced personal ChatGPT accounts on corporate devices?
  • [ ] Is SSO enforced for ChatGPT Enterprise, with SCIM deprovisioning on employee exit?
  • [ ] Is there a BAA in place if PHI may enter ChatGPT? (And controls to ensure it doesn't without one?)
  • [ ] Are connector scopes to SharePoint, Google Drive, GitHub reviewed quarterly?
  • [ ] Do employees see a DLP warning when pasting sensitive data into ChatGPT — or is it silently allowed?
  • [ ] Can your security team see an audit log of every sensitive-data attempt?
  • [ ] Is there a documented AI acceptable use policy, distributed to all employees, signed annually?
  • [ ] Are downstream AI tools (Copilot, Claude, Gemini, Perplexity) covered by the same controls?
  • [ ] Is there a response plan for a ChatGPT-related incident (account takeover, data leak, prompt injection)?

Where ChatGPT Security Is Heading in 2026 and Beyond

Three shifts worth watching:

Agentic AI moves the attack surface. As ChatGPT, Claude, and Copilot gain autonomous agent capabilities that call tools, connect to data sources, and take actions, the risk stops being "what did the employee paste" and becomes "what did the agent decide to send, on its own." Inline inspection at the tool/MCP boundary becomes the new DLP surface.

Shadow AI becomes the real governance problem. Enterprise buyers are slowly realizing that buying ChatGPT Enterprise, Copilot, and Claude Enterprise doesn't eliminate AI risk — it consolidates maybe 40% of it. The rest lives in personal accounts, browser-based agents, and locally-run models. Discovery and enforcement on the endpoint is where the 2026–2027 tooling battle will be fought.

Regulation catches up. The EU AI Act, updated NIST AI RMF, ISO 42001, and state-level laws (California, Colorado, Texas) are converging on a common requirement: documented controls over what sensitive data is sent to what AI systems, with auditable evidence. "We trust OpenAI's ZDR" will not be a sufficient answer.

Strac is building toward all three: agentic DLP via MCP inspection, shadow AI discovery on endpoints, and auditable controls that map to AI governance frameworks.

Conclusion

ChatGPT security in enterprise isn't about whether OpenAI is secure — they are, by any reasonable measure. It's about the gap between what OpenAI protects (the pipes) and what your enterprise needs to protect (the content and the context).

Every meaningful ChatGPT security incident since 2023 has happened in that gap. Employees pasted data they shouldn't have. Personal accounts got breached. Integrations were over-scoped. Agents exfiltrated conversations. None of those are problems OpenAI can solve from their side of the wire.

Strac sits in the gap. Detect sensitive data before it leaves the browser. Redact it before it reaches an integration. Discover shadow accounts before an audit does. Block what shouldn't be there — warn when context matters — audit silently when it's time to learn, not enforce.

Adopt AI aggressively. Govern what your people share with it ruthlessly. That's the combination that wins.

Book a 15-minute demo to see Strac's ChatGPT DLP in action. Or explore the Strac ChatGPT DLP integration, browse all 50+ Strac integrations, or dig deeper on related coverage: Claude DLP, Microsoft Copilot DLP, Slack DLP, Browser DLP, and Endpoint DLP.

Related reading: Does ChatGPT Save Your Data? · MCP DLP · Data Encryption · How Do Companies Protect Customer Data · HIPAA Compliance · SOC 2 Compliance.

Frequently Asked Questions

Is ChatGPT safe to use at work?

Conditionally. ChatGPT Enterprise and the API (with data controls enabled) offer genuine enterprise-grade security — Zero Data Retention, SOC 2 Type II, BAAs, SSO, audit logs. ChatGPT Free, Plus, and Team do not; they train on your data by default and lack enterprise controls. But even ChatGPT Enterprise doesn't stop employees from pasting sensitive data into prompts — that's your responsibility. The answer to "is ChatGPT safe for work" depends entirely on which tier you're on and what content-level controls you've layered on top.

What is the biggest ChatGPT security risk for enterprises?

Employees pasting sensitive data into prompts on personal or unmanaged accounts. Cyberhaven's research found confidential data entering ChatGPT ~200 times per week at an average 100,000-person company. Every significant publicly-disclosed ChatGPT data leak — Samsung, the 225K infostealer credential dump, multiple regulatory investigations — traces back to this root cause. Model-theoretic risks (data reconstruction, poisoning) get more press but cause almost no real enterprise incidents.

Has ChatGPT been hacked?

There has not been a successful breach of OpenAI's core infrastructure that exposed enterprise data at scale. The closest incidents: a March 2023 Redis bug that briefly exposed chat titles and some billing data to other users during a nine-hour window, and ongoing account-takeover attacks (225,000+ credentials on the dark web via infostealer malware, not a ChatGPT breach per se). Prompt injection vulnerabilities in features like ChatGPT Memory and Search have been responsibly disclosed and patched. The bigger practical risk is account-level compromise, not infrastructure-level breach.

Does ChatGPT Enterprise keep my data private?

Yes, within documented limits. ChatGPT Enterprise enables Zero Data Retention by default on prompts and responses, does not train on your data, offers SAML SSO, SCIM, audit logs, customer-managed encryption keys (via Enterprise Key Management), and signs BAAs for healthcare customers. OpenAI is SOC 2 Type II certified. What Enterprise does NOT do: inspect the content of prompts to stop sensitive data from being sent in the first place, discover employees using personal ChatGPT accounts on work devices, or enforce your policies on downstream tools. Those require a layered control like Strac.

How can I prevent employees from leaking data to ChatGPT?

Three layers, in order of effort: (1) Deploy a browser-based detection and enforcement tool like Strac that inspects every ChatGPT prompt before submit, with block/warn/audit modes. (2) Discover and block personal ChatGPT, Claude, and Gemini accounts on corporate devices — buying ChatGPT Enterprise doesn't automatically replace shadow accounts, you have to force the swap. (3) Put DLP in the SaaS tools upstream of ChatGPT (Slack, Zendesk, Jira, Google Drive, SharePoint) so sensitive data is redacted before it ever flows into an AI integration. A policy document alone does not prevent leaks — controls do.

Discover & Protect Data on SaaS, Cloud, Generative AI
Strac provides end-to-end data loss prevention for all SaaS and Cloud apps. Integrate in under 10 minutes and experience the benefits of live DLP scanning, live redaction, and a fortified SaaS environment.
Users Most Likely To Recommend 2024 BadgeG2 High Performer America 2024 BadgeBest Relationship 2024 BadgeEasiest to Use 2024 Badge
Trusted by enterprises
Discover & Remediate PII, PCI, PHI, Sensitive Data

Latest articles

Browse all

Get Your Datasheet

Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.
Close Icon