Generative AI (GenAI)

Proper use of GenAI can improve business efficiencies, customer service, and even quality of life through healthcare and health-related applications. Many will argue that the benefits of GenAI outweigh the risks, but those risks should be considered and mitigated as much as possible.

The cybersecurity risks associated with using Generative AI (GenAI) are multifaceted and stem from both the inherent characteristics of the technology and the ways in which it is deployed and utilized. These risks can broadly be categorized into several key areas:

• Privacy and data protection: GenAI systems require access to vast amounts of data that most users cannot control. Some of this data may include sensitive or personal information that might be publicly shared in response to a request. 
• Input and output: Threat actors have used the request/prompt process of GenAI LLMs to inject malicious data or exploit vulnerabilities in the system. This can cause a data breach if the system is purpose-built for a controlled use, like healthcare or finance. GenAI can also respond to requests with outputs that are inappropriate and harmful. These outputs can be influenced by training, parameters, and malicious action by threat actors. Most users will not know what caused the output.
• Compliance and legal risks: GenAI can complicate compliance with data protection and privacy regulations. For example, the European Union's General Data Protection Regulation (GDPR) imposes strict requirements on the processing of personal data, and GenAI's data handling practices could potentially conflict with these regulations.
• Automated Social Engineering attacks: GenAI is designed to imitate human communication styles, making it a perfect tool for sophisticated phishing and social engineering campaigns. Malicious actors could leverage GenAI to automate the creation of highly convincing phishing emails or messages, making it harder for individuals to distinguish between legitimate and fraudulent communications. Several LLMs have already been stolen and repurposed for malicious use:
  • FraudGPT: A subscription based malicious GenAI tool that creates content for cyberattacks like phishing and impersonation. operates similarly to OpenAI's ChatGPT but lacks the built-in controls and limitations that prevent misuse.
  • WormGPT: An open-source system that is designed to help criminals write malware and malicious code, create phishing content, and find system vulnerabilities.
  • PoisonGPT: This tool spreads misinformation online by inserting false details into political and historical narratives, creating fake news, and manipulating public opinion.
  • XXXGPT: This application was developed to help criminals deploy botnets, malware, keyloggers, infostealers, remote access trojans, and cryptostealers.

Generative AI also creates new risks when it is deployed with an application programming interface (API). APIs enable the integration of separate technologies and are required components in many of the use-cases described above. APIs can create significant risk across the organization. These must be managed and secured like any other component in the digital infrastructure.