If you work with generative AI, sooner or later you’ll face an uncomfortable question: Where does AI security end—and its structural limitations begin? That debate is now at the center of attention following allegations by a cybersecurity researcher who claimed to find vulnerabilities in Microsoft Copilot, and Microsoft’s counterargument denying these are actual security issues.
At TecnetOne, we’re breaking down what happened, why it matters, and what lessons this case holds for any organization using or planning to use AI assistants in professional settings.
The Origin of the Controversy: Four Questionable “Vulnerabilities”
It all started when cybersecurity engineer John Russell published a post claiming he had identified four security issues in Microsoft Copilot. According to him, these flaws allowed risky behaviors in the AI assistant.
However, Microsoft dismissed the reports, arguing they didn't qualify as vulnerabilities under its official criteria. This sparked debate: Are these actual threats—or known limitations of large language models (LLMs)?
Russell reported the following issues:
- Direct and indirect prompt injection leading to exposure of the system prompt.
- Bypassing file upload restrictions using base64-encoded files.
- Executing commands within Copilot’s isolated Linux environment.
While all points were debated, the second one—file bypassing—drew the most attention.
File Upload Bypass: When Rules Can Be Circumvented
Like many AI assistants, Copilot restricts certain file types for security reasons. But Russell demonstrated that encoding a restricted file in base64 and uploading it as plain text allowed it to bypass these policies.
Once inside the session, the file could be reconstructed and analyzed—effectively bypassing Copilot’s upload filters.
This raises a key technical question:
If you can circumvent a restriction meant to protect the system, is that a vulnerability—or just expected behavior?
Learn more: Microsoft 365 Copilot: New Design and New Features
Prompt Injection: The Persistent Problem
Prompt injection is not new. It occurs when an AI interprets input data as instructions, altering its intended behavior.
In Copilot’s case, Russell claimed it was possible to extract the system prompt—the internal instructions that guide the assistant’s behavior.
To some researchers, this is critical. To others, it’s inevitable.
One expert pointed to a similar case: a prompt injection hidden in a Word document, which, when read by Copilot, caused it to behave erratically and crash. The malicious prompt was invisible, hidden within the file’s content.
This highlights how easy it is to embed malicious instructions in seemingly harmless files.
Vulnerability or Inherent AI Limitation?
Here’s where things get interesting. Some experts argue these aren’t vulnerabilities in the classic sense—but rather inevitable consequences of how LLMs operate.
A frequently cited argument:
LLMs still struggle to reliably distinguish between instructions and data.
If an AI can’t fully separate what it should execute vs. what it should interpret, prompt injection is nearly impossible to eliminate without sacrificing usability.
From this perspective, Copilot’s behavior reflects a known technical limitation—not a security flaw.
The Uncomfortable Comparison: “Other AIs Block It”
Russell pushed back on that explanation with a strong counterpoint: other AI tools, like Claude by Anthropic, reject the same inputs that Copilot accepts.
For him, this proves the issue isn’t structural—it’s a validation and control problem specific to Microsoft’s implementation.
This adds nuance to the debate:
If one AI blocks a behavior and another doesn’t—can we really say it’s just a model limitation?
What OWASP Says About System Prompt Exposure
The OWASP GenAI project takes a more balanced stance. According to OWASP, exposing the system prompt is not inherently a vulnerability.
The real risk arises if:
- The prompt contains sensitive data
- It's used for enforcing security policies
- It allows bypassing controls or privilege escalation
In other words: seeing the prompt isn't the issue—what matters is what that prompt protects.
OWASP also points out: even without reading the prompt, attackers can infer rules simply by interacting with the AI and observing its responses.
Microsoft’s Official Position
Microsoft evaluated the reports using its public bug bar—criteria that define what counts as an exploitable vulnerability.
According to Microsoft:
- The reported behaviors did not cross defined security boundaries
- The impact was limited to the user’s own session
- There was no unauthorized data access or privilege escalation
Thus, in their view, these were not security vulnerabilities.
The Real Problem: Differing Risk Definitions
This is the heart of the conflict.
For researchers like Russell, any behavior that lets users manipulate AI beyond intended limits is a security concern.
For Microsoft, it's only a vulnerability if it clearly violates a security boundary.
Both views have merit—but they’re based on different premises.
And this disagreement will only grow as AI becomes more integrated into critical business processes.
Similar titles: How does Microsoft Copilot work in Azure?
What This Means for You and Your Business
If you're using Copilot or any other generative AI tools, here are key takeaways:
- AI is not a security control by itself
Never rely on prompts or internal restrictions as reliable defenses. - Prompt injection is an operational risk—even if not a classic bug
It can affect workflows, outcomes, and decision-making. - Validation and context are more important than ever
Treat inputs, files, and content as potentially hostile. - AI security is an architecture problem—not a patch issue
You need holistic design, not just fixes.
The TecnetOne Perspective
At TecnetOne, we see this case as a clear signal of where cybersecurity is heading in the age of AI.
You must now ask:
- How does AI interact with sensitive data?
- What decisions can it automate?
- What happens if an attacker manipulates those decisions?
The question isn’t just whether something is an “official” vulnerability—but what real-world impact it could have on your operations.
Conclusion: A Debate That’s Just Getting Started
The Copilot case shows that AI security is still under construction. Researchers and vendors often speak different languages, and that causes friction.
But that friction is necessary. It forces us to rethink how we define risk, control, and responsibility in systems that can now reason, interpret, and act.
One thing is certain: The line between “AI limitation” and “security vulnerability” will be one of the biggest debates of the decade.
And being ready for it will define whether your AI tools become an asset—or a liability.
