A former security architect demonstrates 15 different ways to break Copilot: "Microsoft is trying, but if we are honest here, we don't know how to build secure AI applications"

Copilot Pro on Windows.

What you need to know

• Former Microsoft security architect Michael Bargury has identified multiple loopholes hackers can leverage to break Copilot and gain access to sensitive data.
• Microsoft had previously announced its plans to pump the brakes on shipping new experiences to Copilot to improve existing ones based on feedback.
• Microsoft recently highlighted several measures it is implementing to address the rising security concerns across its tech stack, including tying a section of top executives' compensation packages to their security deliverables.

---

While at the Black Hat USA 2024 conference, Former Microsoft security architect Michael Bargury showcased multiple exploits that bad actors can leverage to breach Copilot's security guardrails and misuse its capabilities to cause harm. 

Bargury demonstrated multiple ways hackers can leverage their exploits to access sensitive and intricate credentials from users using Copilot. More specifically, the security architect's findings were centered on Microsoft 365 Copilot. For context, it's an AI-powered experience embedded into the Microsoft 365 suite, including Word and Excel. It accesses your data for a tailored user experience and enhanced workflow. 

Privacy and security are among the top concerns among users that prevent the progression of artificial intelligence. Microsoft has security measures in place to protect the user's data while leveraging Microsoft 365 Copilot's capabilities. However, Bargury was able to bypass them.

In one of the demos dubbed LOLCopilot, Bargury deployed a spear-phishing attack on the AI tool, allowing the security expert to access internal emails. Based on the information gathered from the emails, the tool can draft and send mass emails while mimicking the author's writing style to maintain authenticity. 

Perhaps more concerning is that Copilot can be tricked into accessing sensitive data from employees without raising security alerts. Hackers can use prompts that direct the chatbot to withhold references to originating files, ultimately bypassing Microsoft's data protection protocols.  

Microsoft is trying, but if we are honest here, we don't know how to build secure AI applications.

Michael Bargury, Zenity CTO

Per recent reports, attackers are using sophisticated ploys to lure unsuspecting users to their ploys, including AI. This makes it increasingly hard to detect threats. While speaking to Wired, Bargury indicated, "A hacker would spend days crafting the right email to get you to click on it, but they can generate hundreds of these emails in a few minutes."

Microsoft needs to lay more security layers on its top priority 

A hacker with a hoodie looking through large masses of data accessed using Copilot. (Image credit: Windows Central | Designer by Microsoft)

Generative AI has led to the emergence of powerful tools like ChatGPT and Microsoft Copilot, which spot sophisticated and advanced features like image and text generation. Similarly, these tools are seemingly redefining how users interact with the internet. Even a former Google engineer says the company's biggest challenge to its dominance in search is OpenAI's temporary prototype search tool — SearchGPT.

Earlier this year, Microsoft highlighted its plans to halt shipping new experiences to Copilot. The company further indicated that it would use this opportunity to refine and improve existing experiences based on feedback.

Over the past few months, we've seen Microsoft shift its focus toward security and making its top priority. As highlighted by Microsoft CEO Satya Nadella during the company's earnings report for FY24 Q3, "Security underpins every layer of the tech stack, and it's our No. 1 priority."

Microsoft has faced backlash for its cascade of security failures, including its AI-powered Windows Recall feature, which it was forced to recall before it shipped exclusively to Copilot+ PCs.

Despite making security a team effort at the company and tying a section of top executives' compensation packages to their security deliverables, more security flaws abound.

🔥The hottest trending deals🔥

• 🎮 Seagate Xbox Series X|S Card (2TB) | $249.99 at Best Buy (Save $110!)
• 🕹️Xbox Game Pass Ultimate (3-months) | $29.29 at CDKeys (Save $21!)
• 🎮Lenovo Legion Go (Z1 Extreme, 512GB) | $599.99 at Best Buy (Save $100!)
• 💻HP Victus 15.6 Laptop (RTX 4050) | $599 at Walmart (Save $380!)
• 📺Amazon Fire TV Xbox Game Pass bundle | $82.85 at Amazon (Save $37!)
• ⌨️Surface Pro Keyboard + Slim Pen 2 | $112.99 at Best Buy (Save $167!)
• 🕹️Xbox Game Pass Core (12-months) | $42.09 at CDKeys (Save $18!)
• 💻Lenovo Yoga 7i 16 (Core i5) | $699.99 at Lenovo (Save $440!)
• 🖱️Razer DeathAdder V3 Wired Mouse | $50.99 at Best Buy (Save $19!)
• 🕹️Hi-Fi RUSH (PC, Steam) | $7.89 at CDKeys (Save $22!)
• 🖱️Razer Basilisk V3 Wired Mouse | $49.99 at Best Buy (Save $20!)
• 🖥️Lenovo ThinkStation P3 (Core i5 vPro) | $879.00 at Lenovo (Save $880!)
• ⌨️Alienware Pro Wireless Gaming Keyboard | $149.99 at Dell (Save $50!)
• 🕹️No Man's Sky (PC, Steam) | $24.19 at CDKeys (Save $36!)
• 💻Dell XPS 13 (Snapdragon X Elite) | $1,099.99 at Dell (Save $200!)