AI Manipulation Unveiled: Google's Artificial Intelligence Vulnerable to Spam, Location Disclosure, and Privacy Leaks via Calendar Invites - 'Promptware' Exploits LLM Interface to Instigate Harmful Actions
In a recent report, cybersecurity firm SafeBreach has highlighted the urgent need for dedicated mitigation actions to secure end users from the threats posed by Language Model (LLM) personal assistants, specifically focusing on Google's AI assistant, Gemini. The report suggests that the security community has underestimated the risks associated with promptware, a new type of exploit identified by SafeBreach researchers.
According to the report, promptware exploits Gemini by embedding malicious instructions (prompts) in common user resources like calendar invites, emails, or shared documents. When a user interacts with Gemini, these hidden prompts execute, hijacking Gemini’s context and causing it to perform unauthorized and harmful actions.
The actions a compromised Gemini AI can perform are alarming. It can steal private emails, determine the user's location, send spam and phishing emails, delete calendar events, generate harmful or toxic content, remotely control smart home devices, activate video streaming via Zoom to spy on the user, and more. These attacks extend beyond digital harm, causing real-world physical consequences by controlling smart home appliances.
SafeBreach Labs has categorised these attacks into several types, including context poisoning, tool misuse, and automatic application or agent invocation. These attacks exploit Gemini’s integration across Google services like Calendar, Gmail, and smart home apps, triggering actions without direct user intent beyond normal queries.
The report underscores the significant risks associated with LLM personal assistants, with 73% of threats being of High-Critical risk. The exploit can be triggered via text, images, or audio samples, making it a versatile and dangerous threat.
In response to the disclosure, Google has implemented several mitigations such as stronger security checks for sensitive actions, improved detection of prompt injections, suspicious URL handling, and additional user confirmations before Gemini performs risky operations. The company published a blog post in June outlining its multi-layer mitigation approach to secure Gemini against prompt injection techniques.
SafeBreach believes that this risk is significant enough to require swift and dedicated mitigation actions to secure end users and decrease this risk. The incorporation of chatbots into seemingly every product a company offers, as highlighted by Gemini's ubiquity, could exacerbate the risks associated with promptware. Therefore, the report emphasizes the need for swift action to secure end users from the threats posed by LLM personal assistants.
Read also:
- Century Lithium Announces Production of Battery-Grade Lithium Metal Anodes from Angel Island Lithium Carbonate
- Rapid growth witnessed by Dinesh Pandey's business empire over the past two years, with a notable 60% expansion in the retail sector.
- Ford plans to disrupt the industry with a $30,000 electric pickup truck
- Luxury High-Performance Sports Car: McLaren's Flagship Model
