Microsoft: China Using AI-Generated Content to Sow Division in US

China-affiliated threat actors are ramping up the use of AI to influence and sow division in the US and other countries, according to a new report by the Microsoft Threat Analysis Center (MTAC).

The researchers highlighted how Chinese Communist Party (CCP)-affiliated actors are publishing AI-generated content on social media to amplify controversial domestic issues and criticize the current administration in the US.

In one example, the group Storm-1376, which specializes in influence operations, spread conspiratorial narratives about the Hawaii wildfires in August 2023 across multiple social media platforms.

These posts alleged that the US government had deliberately set the fires to test a military-grade “weather weapon.” Storm-1376 used AI-generated images of burning coastal roads and residences to make the content more eye-catching and posted the texts in at least 31 languages.

In another case, the group launched a social media campaign about the Kentucky train derailment, which took place during the Thanksgiving holiday. These posts spread the narrative that the US government may have caused the derailment and is “deliberately hiding something.”

Storm-1376’s influence campaigns also aimed to stoke discord in countries in East Asia. This included using AI-generated memes and images criticizing the Japanese government after Japan began releasing treated radioactive wastewater into the Pacific Ocean August 2023.

These messages were delivered indiscriminately across social media platforms in numerous languages, including Japanese, Korean and English.

The report also noted the growing use of AI-generated people in videos by China-affiliated actors. This included AI-generated news anchors created by third-party tech companies, using Chinese technology company ByteDance’s CapCut tool.

These AI-generated anchors appeared in a variety of campaigns featuring Taiwanese officials in the build up to Taiwan’s Presidential election in January 2024.

China Tests US Voter Fault Lines

MTAC observed the growing use of Chinese “sockpuppets” posing as US voters on social media to solicit opinions on political topics from American citizens.

The researchers believe this is likely to gather intelligence and precision on key voting demographics ahead of the US Presidential election to inform campaigns to influence voters.

These accounts “nearly exclusively” post about divisive US domestic issues, such as drug use, immigration policies and racial tensions.

Going forward, Microsoft expects China cyber and influence actors to work toward targeting a number of key elections taking place later this year, including in India, South Korea and the US, using AI to assist their campaigns.

“While the impact of such content in swaying audiences remains low, China’s increasing experimentation in augmenting memes, videos, and audio will continue – and may prove effective down the line,” the researchers wrote.

North Korea Continues to Generate Revenue Through Cyber Operations

Another area covered in the report was North Korea’s use of cyber-attacks to generate revenue and collect intelligence on perceived adversaries like the US, South Korea and Japan.

In November 2023, Recorded Future’s Insikt Group estimated that North Korean hackers have stolen over $3bn in cryptocurrency since 2017 as the government seeks to circumvent economic sanctions to fund its weapons program.

Microsoft said that three threat actors it tracks, Jade Sleet, Sapphire Sleet and Citrine Sleet, have focused the most on cryptocurrency targets since June 2023.

North Korean threat actors also heavily targeted the IT sector with spear-phishing and software supply chain attacks during the period covered, from June 2023 to January 2024.

For example, the group Diamond Sleet exploited the TeamCity CVE-2023-42793 vulnerability in October 2023 to compromise hundreds of victims in various industries in the US and European countries like the UK, Denmark, Ireland, and Germany.

Additionally, some North Korean cyber-activities had a geopolitical objective, such as countering the trilateral alliance among the US, South Korea and Japan.

Groups like Ruby Sleet, Emerald Sleet and Pearl Sleet frequently targeted organizations in sectors like government, defense, and the media to collect intelligence on such countries, according to the report.

The researchers also highlighted how North Korean actors are continuing to experiment with AI large language models (LLMs) to make their operations more effective.

For example, Microsoft and OpenAI observed Emerald Sleet utilizing LLMs to enhance spear-phishing campaigns targeting Korean Peninsula experts.