Nation-State Hackers Abuse Gemini AI Tool

Nation-state threat actors are frequently abusing Google’s generative AI tool Gemini to support their malicious cyber operations.

An analysis by the Google Threat Intelligence Group (GTIG) highlighted that APT groups from Iran, China, Russia and North Korea are using the large language model (LLM) for a wide range of malicious activity.

Tasks primarily revolve around research, vulnerability exploitation, malware development and creating and localizing content like phishing emails.

The GTIG said it has not observed any original or persistent attempts by nation-state threat actors to use prompt attacks or other AI-specific threats, with the tool primarily used to improve productivity to date.

A “handful” of attempts have been made to bypass Gemini’s safety controls through publicly available jailbreak prompts. However, these attempts failed, with Gemini responding with safety fallback responses and declined to follow the threat actor’s instructions.

“Rather than enabling disruptive change, generative AI allows threat actors to move faster and at higher volume,” the GTIG researchers noted.

However, with the with new AI models and agentic systems emerging daily, they expect threat actors to evolve their use of AI in kind.

How Gemini is Being Abused by Nation-State Actors

Iran

Iranian government-backed actors accounted for the largest Gemini use linked to APT actors.

Over 30% of Iranian APT actors’ Gemini use was linked to APT42, a group observed to target military and political figures in countries such as the US and Israel.

The GTIG said it observed Iranian APT actors using the tool for reconnaissance on potential targets, such as defense experts and organizations, foreign governments and individual dissidents.

They also undertook research into publicly reported vulnerabilities on specific technologies. This included searching for exploitation techniques.

Gemini was also frequently used by Iranian actors to craft legitimate-looking phishing emails. This included using the LLM’s text generation and editing capabilities for translation and tailoring messages for particular sectors and locations.

China

Chinese APT groups used Gemini for reconnaissance purposes, with a particular focus on US military and IT organizations.

Additionally, there was a notable focus on using the tool to assist with compromise. This included scripting and development of malware and finding solutions to technical challenges.

For post-compromise activities, Gemini was used to provide information on enabling deeper access in target networks, such as lateral movement, privilege escalation and data exfiltration.

For example, one PRC-backed group asked Gemini for assistance to figure out how to sign a plugin for Microsoft Outlook and silently deploy it to all computers.

North Korea

The GTIG observed North Korean state actors using Gemini to support several phases of the attack lifecycle.

This included research specifically on how to compromise Gmail accounts and other Google services.

Another core focus was to assists IT worker schemes, in which North Korean actors falsely gain employment with Western IT firms to generate revenue for the Democratic People’s Republic of Korea (DPRK) government.

For these campaigns, Gemini was used to research information like available jobs on LinkedIn and average salary, and to generate content such as cover letters from job postings.

North Korean actors also engaged with Gemini with several questions that appeared focused on conducting initial research and reconnaissance into prospective targets, such as US and South Korean defense contractors.

Some of these APT groups also tried to use Gemini to assist with development and scripting tasks, including developing code for sandbox evasion.

Russia

Russian nation-state groups were more limited in their abuse of Gemini compared to other nations, according to GTIG.

Observed uses included help with rewriting publicly available malware into another language and adding encryption functionality to code.

The GTIG said a possible reason for this low engagement is that Russian actors are avoiding Gemini and other Western-controlled platforms to avoid monitoring of their activities. They may be using AI tools produced by Russian firms or locally hosting LLMs as an alternative.