I Found Out Microsoft Copilot Was Reading Confidential Emails — And It Gets Worse

If you work in enterprise software, this story should make your blood run cold.

Earlier this month, Microsoft quietly acknowledged that Microsoft 365 Copilot Chat has been summarizing emails labeled "confidential" — completely ignoring Data Loss Prevention (DLP) policies that were explicitly configured to prevent it. The bug, tracked as CW1226324, was first reported by customers on January 21, 2026, and Microsoft has only recently started addressing it.

Let that sink in. The AI assistant that millions of enterprise workers rely on daily was actively bypassing security guardrails designed to protect sensitive information.

What Actually Happened

Here is the breakdown: organizations using Microsoft 365 can apply sensitivity labels to emails — things like "Confidential," "Internal Only," or "Highly Restricted." These labels work with DLP policies to ensure sensitive content does not get shared, copied, or processed inappropriately.

Except Copilot decided those rules did not apply to it.

When users opened the Copilot Chat "work tab," the AI would happily summarize confidential emails, surfacing sensitive information in a context where DLP protections were supposed to prevent exactly that. Imagine HR emails about layoffs, M&A discussions, legal strategy documents — all cheerfully summarized by an AI that was supposed to respect boundaries.

It is no wonder that 72 percent of S&P 500 companies now cite AI as a material risk in their regulatory filings.

Why This Matters More Than a Typical Bug

As a QA automation engineer, I see bugs every day. But this one is different for three reasons:

1. It is a trust violation, not just a defect. When an organization configures DLP policies, they are making a security promise to their employees, clients, and regulators. Copilot broke that promise silently. Nobody got an error message. Nobody was warned. The AI just... ignored the rules.

2. It exposes a fundamental testing gap. How do you QA an AI system's compliance with security policies? Traditional test automation validates that features work. But validating that an AI model respects boundaries requires an entirely different approach — one that most QA teams are not equipped for yet.

3. It happened at Microsoft-scale. This is not some startup's beta product. This is Microsoft 365, used by over 400 million people. The blast radius of AI security failures at this scale is enormous.

The Bigger Picture: Gemini 3.1, Anthropic's $30B, and the Race Nobody Is Testing

This Copilot incident lands in a week where Google launched Gemini 3.1 Pro with improved reasoning benchmarks, Anthropic closed a $30 billion Series G at a $380 billion valuation, and OpenAI continues pushing GPT-5.3 Codex. The AI model race is accelerating faster than ever.

But here is what keeps me up at night: the testing and safety infrastructure is not keeping pace with the capabilities. We are shipping increasingly powerful AI systems into enterprise environments while the guardrails — DLP, sensitivity labels, access controls — were designed for a pre-AI world.

What QA Engineers Should Be Doing Right Now

If you are in QA or security testing, this incident is your wake-up call:

• Test AI boundary compliance explicitly. Do not assume that because a policy is configured, the AI respects it. Write tests that verify AI outputs against sensitivity labels.
• Automate DLP validation. Tools like Playwright can be used to simulate Copilot interactions and verify that confidential content is not surfaced.
• Red-team your AI integrations. Treat every AI assistant like an untrusted third party. What data can it access? What should it not be able to summarize?
• Monitor AI outputs in production. Logging and observability for AI-generated summaries should be table stakes, especially in regulated industries.

The Bottom Line

We are in an era where AI systems are being granted access to our most sensitive data — and the security models have not caught up. The Microsoft Copilot DLP bypass is not an isolated incident. It is a preview of the AI security challenges every enterprise will face as these tools become embedded in daily workflows.

The companies that invest in AI-aware QA and security testing now will be the ones that survive the next wave. The ones that blindly trust their AI vendors? Well, their confidential emails might already be getting summarized somewhere they should not be.

---

Want to discuss AI security testing strategies? Connect with me on LinkedIn or X.