Malicious email summaries with warnings directing to phishing sites could be generated using Google Gemini for Workspace, BleepingComputer reports.
Attacks commence with the creation of an email with a concealed order, which would be parsed and obeyed by Gemini upon the victim's prompting of an email summary generation, according to Mozilla GenAI Bug Bounty Programs Manager Marco Figueroa, who reported the issue through Mozilla's 0din program. Such a new attack technique could be combated either through the removal and disregard of messages hidden in the email or the adoption of a post-processing filter that would flag potentially malicious content within Gemini-generated summaries, said Figueroa. Meanwhile, Google has noted that there has been no indication of Gemini manipulation as discovered by Figueroa, while emphasizing its defenses against prompt injection intrusions. "We are constantly hardening our already robust defenses through red-teaming exercises that train our models to defend against these types of adversarial attacks," said a Google spokesperson.
Attacks commence with the creation of an email with a concealed order, which would be parsed and obeyed by Gemini upon the victim's prompting of an email summary generation, according to Mozilla GenAI Bug Bounty Programs Manager Marco Figueroa, who reported the issue through Mozilla's 0din program. Such a new attack technique could be combated either through the removal and disregard of messages hidden in the email or the adoption of a post-processing filter that would flag potentially malicious content within Gemini-generated summaries, said Figueroa. Meanwhile, Google has noted that there has been no indication of Gemini manipulation as discovered by Figueroa, while emphasizing its defenses against prompt injection intrusions. "We are constantly hardening our already robust defenses through red-teaming exercises that train our models to defend against these types of adversarial attacks," said a Google spokesperson.
