OpenAI has taken decisive action against several malicious actors, banning ChatGPT accounts linked to Russian, Iranian, and Chinese state-sponsored hacking groups, as well as other nefarious activities.
The AI firm revealed that these accounts were leveraged for a range of illicit purposes, including malware development, social media manipulation, and sensitive research.
One prominent case involves a Russian-speaking threat actor, codenamed “ScopeCreep” by OpenAI, who used ChatGPT to refine Windows malware, debug code across multiple languages, and establish command-and-control infrastructure.
OpenAI noted the actor’s sophisticated understanding of Windows internals and strong operational security (OPSEC) behaviors.
This group employed a tactic of using temporary email accounts to create new ChatGPT accounts, each used for a single conversation to incrementally improve their malicious software before being abandoned.
The AI-assisted malware developed by ScopeCreep was then distributed through a public code repository, deceptively impersonating a legitimate video game crosshair overlay tool called Crosshair X.
Users who downloaded this trojanized software had their systems infected with a malware loader designed to retrieve additional payloads, escalate privileges, establish persistence, exfiltrate sensitive data, and evade detection.
The malware specifically sought to harvest credentials, tokens, and cookies from web browsers and could alert threat actors via a Telegram channel upon new victim compromises.
OpenAI’s models were used by ScopeCreep to debug Go code snippets, integrate Telegram API, and modify Windows Defender settings.
In addition to the Russian-speaking threat actor, OpenAI disabled ChatGPT accounts associated with two Chinese state-sponsored hacking groups: APT5 (also known as Bronze Fleetwood, Keyhole Panda, Manganese, and UNC2630) and APT15 (also known as Flea, Nylon Typhoon, Playful Taurus, Royal APT, and Vixen Panda).
These groups engaged with ChatGPT for open-source research on various entities and technical topics, script modification, and troubleshooting system configurations.
Their activities also included Linux system administration, software development, and infrastructure setup, such as building software packages for offline deployment and seeking advice on firewalls and name servers.
The China-linked clusters further utilized ChatGPT for developing brute-force scripts to breach FTP servers, researching the use of large-language models (LLMs) for automated penetration testing, and creating code to manage Android devices for programmatic social media posting and liking on platforms like Facebook, Instagram, TikTok, and X.
OpenAI’s report also detailed other observed malicious activities leveraging ChatGPT.
Sneer Review (likely China-origin) activity generated bulk social media posts in English, Chinese, and Urdu on geopolitically relevant topics for sharing on Facebook, Reddit, TikTok, and X.
Operation High Five (Philippines-origin) used OpenAI’s models to generate high volumes of short comments in English and Taglish on Philippine politics and current events for Facebook and TikTok.
Operation VAGue Focus (China-origin) generated social media posts for X, posing as journalists and geopolitical analysts, asking about computer network attack tools, and translating emails from Chinese to English for suspected social engineering attempts.
