China-linked accounts used ChatGPT to help create fake Facebook personas, avoid detection: OpenAI
The chatbot was also asked to assist in conceptualising an AI tool to monitor online opinion and collect “harmful” content from “key persons”, according to OpenAI’s latest threat report.
The OpenAI logo is seen in this illustration taken on May 20, 2024. (File photo: Reuters/Dado Ruvic/Illustration)
New: You can now listen to articles.
This audio is generated by an AI tool.
Read a summary of this article on FAST.
Get bite-sized news via a new
cards interface. Give it a try.
Click here to return to FAST
Tap here to return to FAST
FAST
SINGAPORE: A China-linked influence operation used ChatGPT to help design a Facebook playbook for building credible personas, amplifying narratives and reducing the risk of platform detection, according to OpenAI.
This was flagged in the AI giant’s latest threat report published on Thursday (Jun 11), which detailed how OpenAI had banned likely China-origin ChatGPT accounts for using the popular chatbot to support two covert influence operations.
The United States tech firm said one of the operations was behind a data centre-focused campaign and likely run by a private Chinese technology company conducting work for Chinese provincial-level government clients. The accounts here generated content claiming that the rapid construction of data centres for AI was pushing up electricity prices for ordinary American families.
The other operation ran a campaign producing content criticising US tariffs and claiming that the US was seeking technological dominance. Operators here specifically instructed the model to depict only US President Donald Trump in political cartoons and to leave out any imagery of China or Chinese leader Xi Jinping.
Guess Word
Crack the word, one row at a time
Buzzword
Create words using the given letters
Mini Sudoku
Tiny puzzle, mighty brain teaser
Mini Crossword
Small grid, big challenge
Word Search
Spot as many words as you can
They also asked ChatGPT to propose a concept for an AI system to surveil online public opinion, and one account showed a possible link to earlier China-origin activity targeting the Philippines.
OpenAI assessed that both campaigns gained little or no authentic engagement.
“Foreign influence operations have long sought to latch onto existing local issues and sincerely held beliefs, using them to build credibility, amplify divisions or exacerbate public distrust,” said the company.
“In this case, the operators attempted to covertly insert themselves into an ongoing American debate about the future of the country’s AI capabilities while hiding who they were and what motivated them.”
FACEBOOK PLAYBOOK
The accounts associated with the data centre campaign had asked ChatGPT to generate, polish and edit internal work reports, with one such report focused on Facebook, said OpenAI.
It emphasised building “real, trustworthy” and daily-life personas that post lifestyle, current-affairs and professional content, and using cross-account interactions to amplify messaging while preserving the appearance of genuine engagement.
The operators had studied Facebook features like groups, pages, hashtags, advertising tools and recommendation and reporting systems, framing their approach as a “dual-track” combination of organic posting and paid ads. They also emphasised account safety by creating backups and separating account operations to avoid the platform detecting coordination.
The accounts prompted ChatGPT in simplified Chinese and used virtual private networks (VPNs) – which mask a user’s location – to reach the platform, which is not officially available in China.
OpenAI also assessed that the operators behind these accounts were likely part of a social media operations team at a private Chinese technology company doing work for provincial-level government clients.
That arrangement, said OpenAI, appeared consistent with a commercial ecosystem that supports the priorities of China’s Communist Party and government in “public opinion guidance” – in which private firms are contracted to help shape narratives on the state’s behalf.
SCOPING FOR SURVEILLANCE
The other campaign on tech and tariffs went further than content and report generation. Its operators asked ChatGPT to propose a concept for an AI system to surveil online public opinion – one that would automatically collect what they defined as “harmful” information from “key persons” on social media, said OpenAI.
The operators wanted this system to store logs, download videos for automated content analysis and also send risk notifications. OpenAI said its model produced only a general 500-word response on data storage and management, and did not provide ideas on how to collect data for surveillance purposes.
This request bore similarities to a case OpenAI outlined in an October 2025 threat report, in which Chinese government entities sought help drafting an “early warning” system to track the travel of people categorised as Uyghur-related and “high-risk”.
OpenAI also observed content overlap between this tech and tariffs campaign and earlier China-origin activity targeting the Philippines. In 2024, a video that circulated widely on social media purportedly showed Philippine President Ferdinand Marcos Jr “sniffing” illegal drugs – which was eventually debunked by deepfake experts as inauthentic. The October 2025 OpenAI report then found inauthentic accounts reviving this 2024 falsehood to criticise Marcos.
The overlap stemmed from a single X account who posted a cartoon slamming Trump’s tariffs and had also shared the Marcos material earlier.
OpenAI noted however that this was insufficient to conclusively establish a connection, though it reinforced the impression that there was a “nexus of activity” on X aimed at amplifying influence efforts from China.
OpenAI said both campaigns sit within a broader pattern of AI misuse originating from China that it has disrupted over recent years. In February, the firm flagged that an account linked to Chinese law enforcement had tried to use ChatGPT to plan an influence operation against Japanese Prime Minister Sanae Takaichi.
OpenAI noted that the campaigns had occurred against the backdrop of escalating US-China economic and technology competition, and following a key Communist Party meeting that elevated AI as a strategic priority under China’s next five-year plan.
In concluding its report, the tech firm said it was “ironic” that the operators behind the campaigns had used American AI – rather than Chinese models – to generate content attacking American AI.
“We are not in a position to determine what drove this choice,” said OpenAI.
Source: CNA/jo
Sign up for our newsletters
Get the CNA app
Stay updated with notifications for breaking news and our best stories
Get WhatsApp alerts
Join our channel for the top reads for the day on your preferred chat app
Get bite-sized news via a new
cards interface. Give it a try.
Click here to return to FAST
Tap here to return to FAST
FAST
About The Author
