Nation-State Hackers Increasingly Leverage Google’s Gemini AI for Cyberattacks

Cybercriminals, particularly state-sponsored threat actors from nations including North Korea, Iran, China, and Russia, are actively misusing Google’s Gemini large language model (LLM) to enhance and accelerate various stages of the cyberattack lifecycle. This trend, documented by Google’s Threat Intelligence Group (GTIG), signals a rapidly evolving threat landscape where artificial intelligence is becoming a key tool for malicious actors.

Gemini’s Role in the Attack Lifecycle

GTIG’s research reveals that threat actors are utilizing Gemini for a wide range of tasks, including coding and scripting, accelerating reconnaissance, researching publicly known vulnerabilities, and aiding in malware development and post-compromise activities. The accessibility and capabilities of Gemini are lowering the barrier to entry for sophisticated cyberattacks.

Specific Examples of State-Sponsored Activity

North Korea

The North Korean government-backed group UNC2970 has been observed using Gemini to synthesize open-source intelligence (OSINT) and profile high-value targets. This activity supports campaign planning and reconnaissance efforts, allowing the group to identify and assess potential targets more efficiently. Source

Iran

APT42, an Iranian-sponsored threat actor, is leveraging generative AI models, including Gemini, to locate official email addresses associated with specific entities. This reconnaissance is a crucial step in phishing operations, enabling targeted attacks against individuals within organizations. Source

The Rise of Model Extraction Attacks

Beyond direct use in attacks, GTIG has noted a significant increase in model extraction attacks, also known as distillation attacks. These attacks, primarily originating from private sector entities, aim to accelerate AI model development by extracting information from existing models like Gemini to train new ones at a lower cost. Organizations offering AI models as a service must closely monitor API access for signs of such activity. Source

AI-Integrated Malware and Underground Ecosystems

While fully autonomous AI-enabled attacks haven’t yet materialized, threat actors are increasingly incorporating AI-generated capabilities into their existing intrusion operations. For example, the HonestCue malware utilizes Gemini’s API to dynamically generate and execute malicious C# code in memory, enhancing its functionality and evasion capabilities. Source

an underground “jailbreak” ecosystem is emerging, providing tools and services to bypass restrictions on AI models and facilitate malicious activities. Xanthorox, marketed as an autonomous AI platform for generating phishing content, malware, and ransomware, was found to be powered by third-party commercial AI products, including Gemini, demonstrating a reliance on existing AI infrastructure rather than custom model development.

Looking Ahead

The growing misuse of generative AI like Gemini underscores the necessitate for heightened vigilance and proactive security measures. Organizations should strengthen safeguards, monitor AI platform usage, and continuously test their security posture to adapt to increasingly sophisticated, AI-driven adversaries. The evolving threat landscape demands a proactive and adaptive approach to cybersecurity.