A new security report by Kaspersky sheds light on vulnerabilities in ChatGPT, the widely used language model, revealing a surge in AI-related crimes and exploitations. The report highlights instances of malware creation, the sale of ChatGPT jailbreak prompts in shadow forums, and the potential for sophisticated attacks as language models continue to evolve.

Key Points:

1. AI Crime Surge:
   • The report covers the period between January and December 2023, indicating a significant increase in the use of GPT (Generative Pre-trained Transformer) for creating malware within the ChatGPT domain. While no instances of such malware have emerged, the threat remains.
2. Malicious Code Generation:
   • ChatGPT is shown to be utilized for generating code to address challenges encountered during the theft of user data. This showcases how ChatGPT can be leveraged to lower the entry barrier for individuals with malicious intent, enabling them to exploit AI’s capabilities for nefarious purposes.
3. Jailbreak Prompts for Sale:
   • The report reveals the existence of ChatGPT jailbreak prompts available for purchase, with 249 offers found during 2023. These prompts, often shared on social platforms and shadow forums, allow users to circumvent ChatGPT’s safety measures. While not all prompts are designed for illegal activities, some facilitate more precise results.
4. Security Risks and Exploitation:
   • Exploiting ChatGPT’s vulnerabilities extends beyond illegal activities, with potential applications in penetration testing scenarios. The report mentions the use of basic Language Model (LLM) in “evil” modules focused on illicit activities.
5. AI-Related Posts and Stolen Accounts:
   • The report notes a rise in AI-related posts, including discussions around creating duplicitous or “evil” ChatGPT prompts for phishing and stealing user data. Additionally, there has been an illicit trade of stolen ChatGPT Plus accounts on the dark web.
6. Warning of Sophisticated Attacks:
   • Kaspersky concludes the report with a warning about the increasing capabilities of language models, predicting that as these capabilities advance, sophisticated attacks may become possible.

As the use of ChatGPT continues to grow, the report emphasizes the need for vigilance in safeguarding personal information and underscores the evolving landscape of AI-related security threats.