OpenAI: ChatGPT weaponized in Chinese influence campaign

OpenAI has disclosed that Chinese law enforcement exploited its ChatGPT chatbot to deploy influence operations against Japanese Prime Minister Sanae Takaichi and other Chinese Communist Party critics in October, according to The Register.

Multiple prompts seeking publication and amplification of negative comments against Takaichi following her remarks on the CCP's alleged human rights violations in Inner Mongolia have been rejected by ChatGPT, prompting the Chinese police-linked user to leverage other LLMs, noted a report from OpenAI. However, ChatGPT was later asked by the user to generate status reports on the attack against Takaichi, as well as other operations against Chinese dissidents.

Analysis of the generated status reports revealed that the attacker had pursued Japanese influence support, while deploying multiple hashtags, none of which gained a significant audience. OpenAI also discovered in the user's cyber ops diary other campaigns that wield psychological pressure against critics online.

"This is what Chinese, modern, transnational repression looks like. It's not just digital. It's not just about trolling. It's about trying to hit critics of the CCP with everything everywhere, all at once," said OpenAI Intelligence and Investigations Team lead investigator Ben Nimmo.