For more than 20 years, Open Worldwide Application Security Project (OWASP) top 10 risk lists has have been go-to references in the fight to make software more secure. In 2023, OWASP brought forward a new addition: a rundown of risks specific to AI. Two draft versions of the AI risk list were published in spring/summer of that year, with a formal version 1 released in October.
Since then, LLMs have only become more entrenched as business productivity tools. Most companies are either using or exploring the use of AI, and while some liabilities are well known—such as the need to always check an LLM’s work—others remain under the radar.
We did some analysis and found the vulnerabilities identified by OWASP fall broadly into three categories:
- Access risks associated with exploited privileges and unauthorized actions.
- Data risks such as data manipulation or loss of services.
- Reputational and business risks resulting from bad AI outputs or actions.
In this blog, we take a closer look at the specific risks in each case and offer some suggestions about how to handle them.
1. Access risks with AI
Of the 10 vulnerabilities listed by OWASP, three are specific to access and misuse of privileges: insecure plugin design, insecure output handling, and excessive agency.
According to OWASP, an LLM using that uses insecure could lose access control, opening them up to malicious requests or the execution of unauthorized remote code. On the flipside, plugins or applications that handle large language model outputs insecurely—without evaluating them—could expose backend systems be susceptible to XSS, CSRF, and SSRF attacks that execute unwanted actions, and to unauthorized privilege escalations, and remote code execution.
And because AI chatbots are ‘actors’ able to make and implement decisions, it matters how much free reign (i.e., agency) they’re given. As OWASP explains, “Excessive Agency is the vulnerability that enables damaging actions to be performed in response to unexpected/ambiguous outputs from an LLM (regardless of what is causing the LLM to malfunction; be it hallucination/confabulation, direct/indirect prompt injection, malicious plugin, poorly-engineered benign prompts, or just a poorly-performing model).”
For example, a personal mail reader assistant with message-sending capabilitiess could be exploited by a malicious email to propagate spam from a user’s account.
In all these cases, the large language model becomes a conduit for bad actors to infiltrate systems.
2. AI and data risks
Poisoned training data, supply chain vulnerabilities, sensitive information disclosures, prompt injection vulnerabilities , and denials of service are all data-specific AI risks.
Data can be poisoned deliberately by bad actors and inadvertently when an AI system learns from unreliable or unvetted sources. Both types of poisoning can occur within an active AI chatbot application or emerge from the LLM supply chain, where reliance on pre-trained models, crowdsourced data, and insecure plugin extensions may produce biased data outputs, security breaches, or system failures.
Poisoned data and the supply chain are input concerns. Allowing private, confidential, personally identifying information and the like into model training data can also result in unwanted disclosures of sensitive information.
With prompt injections, ill-meaning inputs may cause a large language model AI chatbot to expose data that should be kept private or perform other actions that lead to data compromises.
AI denial of service attacks are similar to classic DOS attacks. They may aim to overwhelm a large language model and deprive users of access to data and apps, or—because many AI chatbots rely on pay-as-you-go IT infrastructure—force the system to consume excessive resources and rack up massive costs.
3. Reputational and business risks associated with AI
The final two OWASP vulnerabilities relate to model theft and overreliance on AI. The first applies when an organization has its own proprietary LLM model. If that model is accessed, copied, or exfiltrated by unauthorized users, it could be exploited to harm the performance of a business, disadvantage it competitively, and potentially cause a leak of sensitive information.
Overreliance on AI is already having consequences around the world today. There’s no shortage of stories about large language models generating false or inappropriate outputs from fabricated citations and legal precedents to racist and sexist language.
OWASP points out that depending on AI chatbots without proper oversight can make organizations vulnerable to publishing misinformation or offensive content that results in reputational damage or even legal action.
Given all these various risks, the question becomes, “What can we do about it?” Fortunately, there are some protective steps organizations can take.
What enterprises can do about AI vulnerabilities
From our perspective at Trend Micro, defending against AI access risks requires a zero-trust security stance with disciplined separation of systems (sandboxing). Even though generative AI can challenge zero-trust defenses in ways that other IT systems don’t—because it can mimic trusted entities—a zero-trust posture still adds checks and balances that make it easier to identify and contain unwanted activity. OWASP also advises that large language models “should not self-police” and calls for controls to be embedded in application programming interfaces (APIs).
Sandboxing is also key to protecting data privacy and integrity: keeping confidential information fully separated from shareable data and making it inaccessible to AI chatbots and other public-facing systems.
Good separation of data prevents large language models from including private or personally identifiable information in public outputs, and from being publicly prompted to interact with secure applications such as payment systems in inappropriate ways.
On the reputational front, the simplest remedies are to not rely solely on AI-generated content or code, and to never publish or use AI outputs without first verifying they are true, accurate, and reliable.
Many of these defensive measures can—and should—be embedded in corporate policies. Once an appropriate policy foundation is in place, security technologies such as endpoint detection and response (EDR), extended detection and response (XDR), and security information and event management (SIEM) can be used for enforcement and to monitor for potentially harmful activity.
Large language model AI chatbots are here to stay
OWASP’s catalogue of AI risks proves that concerns about the rush to embrace AI are well justified. At the same time, AI clearly isn’t going anywhere, so understanding the risks and taking responsible steps to mitigate them is critically important.
Setting up the right policies to manage AI use and implementing those policies with the help of cybersecurity solutions is a good first step. So is staying informed. The way we see it at Trend Micro, OWASP’s top 10 AI risk list is bound to become as much of an annual must-read as its original application security list has been since 2003.
