{"id":6924,"date":"2025-01-15T09:00:00","date_gmt":"2025-01-15T15:00:00","guid":{"rendered":"https:\/\/www.darkreading.com\/vulnerabilities-threats\/owasps-llm-top-10-shows-emerging-ai-threats"},"modified":"2025-01-15T09:00:00","modified_gmt":"2025-01-15T15:00:00","slug":"owasps-new-llm-top-10-shows-emerging-ai-threats","status":"publish","type":"post","link":"https:\/\/ddi.mohflo.net\/index.php\/2025\/01\/15\/owasps-new-llm-top-10-shows-emerging-ai-threats\/","title":{"rendered":"OWASP’s New LLM Top 10 Shows Emerging AI Threats"},"content":{"rendered":"
<\/a><\/div>\n
<\/div>\n

COMMENTARY<\/span><\/span><\/p>\n

The advent of artificial intelligence (AI) coding tools undoubtedly signifies a new chapter in modern software development. With <\/span>63% of organizations<\/a><\/span> currently piloting or deploying AI coding assistants into their development workflows, the genie is well and truly out of the bottle, and the industry must now make careful moves to integrate it as safely and efficiently as possible.<\/span><\/p>\n

The OWASP Foundation has long been a champion of secure coding best practices, providing extensive coverage on how developers can best defend their codebases from exploitable vulnerabilities. Its recent update to the <\/span>OWASP Top 10 for Large Language Model (LLM) Applications<\/a><\/span> reveals the emerging and most potent threats perpetuated by AI-generated code and generative AI (GenAI) applications, and this is an essential starting point for understanding and mitigating the threats likely to rear their ugly head. <\/span><\/p>\n

We must focus on integrating solid, foundational controls around developer risk management if we want to see more secure, higher quality software in the future, not to mention make a dent in the flurry of global guidelines that demand applications are released that are <\/span>secure by design<\/a><\/span>.<\/span><\/p>\n

The Perilous Crossover Between AI-Generated Code and Software Supply Chain Security<\/h2>\n

Prompt Injection’s ranking as the No. 1 entry on the latest OWASP Top 10 was unsurprising, given its function as a direct natural language command telling the software what to do (for better or worse). However, Supply Chain Vulnerabilities, which have a much more significant impact at the enterprise level, came in at No. 3. <\/span><\/p>\n

OWASP’s advice mentions several attack vectors comprising this category of vulnerability, elements such as implementing pretrained models that are also precompromised with backdoors, malware and poisoned data, or <\/span>vulnerable LoRA adapters<\/a><\/span> that, ironically, are used to increase efficiency, but can, in turn, compromise the base LLM. These present potentially grave, widespread exploitable issues that can permeate the whole supply chain in which they are used.<\/span><\/p>\n

Sadly, many developers are not skill- and process-enabled enough to navigate these problems safely, and this is even more apparent when assessing AI-generated code for business logic flaws. While not specifically listed as a category, as is apparent in <\/span>OWASP\u2019s Top 10 Web<\/a><\/span> Application Security Risks<\/a><\/span>, this is partly covered in No. 6, Excessive Agency. Often, a developer will vastly overprivilege the LLM for it to operate more seamlessly, especially in testing environments, or misinterpret how real users will interact with the software, leaving it vulnerable to exploitable logic bugs. These, too, affect supply chain applications and, overall, require a developer to apply critical thinking and threat modeling principles to overcome them. Unchecked AI tool use, or adding AI-powered layers to existing codebases, adds to the overall complexity and is a significant area of developer-driven risk.<\/span><\/p>\n

Data Exposure Is a Serious Concern Requiring Serious Awareness<\/h2>\n

Sensitive Information Disclosure is second on the new list, but it should be a chief concern for enterprise security leaders and development managers. As OWASP points out, this vector can affect both the LLM itself and its application context, leading to <\/span>personally identifiable information (PII) exposure<\/a><\/span>, and disclosure of proprietary algorithms and business data.<\/span><\/p>\n

The nature of how the technology operates can mean that exposing this data is as simple as using cunning prompts rather than actively “hacking” a code-level vulnerability, and “<\/span>the grandma exploit<\/a><\/span>” is a prime example of sensitive data being exposed due to lax security controls over executable prompts. Here, ChatGPT was duped into revealing the recipe for napalm when prompted to assume the role of a grandmother reading a bedtime story. <\/span>A similar technique<\/a><\/span> was also used to extract <\/span>Windows 11 keys<\/a><\/span>. <\/span><\/p>\n

Part of the reason this is made possible is through poorly configured model outputs that can expose proprietary training data, which can then be leveraged in inversion attacks to eventually circumvent the security controls. This is a high-risk area for those who are feeding training data into their own LLMs, and the use of the technology requires companywide, role-based security awareness upskilling. The developers building the platform must be well-versed in input validation and data sanitization (as in, these skills are verified and assessed before they can commit code), and <\/span>every<\/span><\/span> end user must be trained to avoid feeding sensitive data that can be spat out at a later date.<\/span><\/p>\n

While this may seem trivial on a small scale, at the government or enterprise level, with the potential for tens of thousands of employees to inadvertently participate in exposing sensitive data, it’s a significant expansion of an already unwieldy attack surface that must be addressed.<\/span><\/p>\n

Are You Paying Attention to Retrieval-Augmented Generation (RAG)?<\/h2>\n

Perhaps the most notable new entry in the 2025 list is featured at No. 8, Vector and Embedding Weaknesses. With enterprise LLM applications often utilizing RAG technology as part of the software architecture, this is a vulnerability category to which the industry must pay close attention.<\/span><\/p>\n

RAG is essential for model performance enhancement, often acting as the “glue” that provides contextual cues between pre-trained models and external knowledge sources. This is made possible by implementing vectors and embeddings, but if they are not implemented securely they can lead to <\/span>disastrous data exposure<\/a><\/span>, or pave the way for serious data poisoning and embedding inversion attacks. <\/span><\/p>\n

A comprehensive understanding of both core business logic and least-privilege access control should be considered a security skills baseline for developers working on internal models. However, realistically, the best-case scenario would involve utilizing the highest-performing, security-skilled developers and their AppSec counterparts to perform comprehensive threat modeling and ensure sufficient logging and monitoring.<\/span><\/p>\n

As with all LLM technology, while this is a fascinating emerging space, it should be crafted and used with a high level of security knowledge and care. This list is a powerful, up-to-date foundation for the current threat landscape, but the environment will inevitably grow and change quickly. The way in which developers create applications is sure to be augmented in the next few years, but ultimately, there is no replacement for an intuitive, security-focused developer working with the critical thinking required to drive down the risk of both AI and human error.<\/span><\/p>\n

Source<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"

COMMENTARY The advent of artificial intelligence (AI) coding tools undoubtedly<\/p>\n","protected":false},"author":12,"featured_media":6925,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[1],"tags":[809],"class_list":["post-6924","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-uncategorized","tag-dark-reading"],"featured_image_urls":{"full":["https:\/\/i0.wp.com\/ddi.mohflo.net\/wp-content\/uploads\/2025\/01\/owasps-new-llm-top-10-shows-emerging-ai-threats.jpg?fit=1800%2C1013&ssl=1",1800,1013,false],"thumbnail":["https:\/\/i0.wp.com\/ddi.mohflo.net\/wp-content\/uploads\/2025\/01\/owasps-new-llm-top-10-shows-emerging-ai-threats.jpg?resize=150%2C150&ssl=1",150,150,true],"medium":["https:\/\/i0.wp.com\/ddi.mohflo.net\/wp-content\/uploads\/2025\/01\/owasps-new-llm-top-10-shows-emerging-ai-threats.jpg?fit=300%2C169&ssl=1",300,169,true],"medium_large":["https:\/\/i0.wp.com\/ddi.mohflo.net\/wp-content\/uploads\/2025\/01\/owasps-new-llm-top-10-shows-emerging-ai-threats.jpg?fit=640%2C360&ssl=1",640,360,true],"large":["https:\/\/i0.wp.com\/ddi.mohflo.net\/wp-content\/uploads\/2025\/01\/owasps-new-llm-top-10-shows-emerging-ai-threats.jpg?fit=640%2C360&ssl=1",640,360,true],"1536x1536":["https:\/\/i0.wp.com\/ddi.mohflo.net\/wp-content\/uploads\/2025\/01\/owasps-new-llm-top-10-shows-emerging-ai-threats.jpg?fit=1536%2C864&ssl=1",1536,864,true],"2048x2048":["https:\/\/i0.wp.com\/ddi.mohflo.net\/wp-content\/uploads\/2025\/01\/owasps-new-llm-top-10-shows-emerging-ai-threats.jpg?fit=1800%2C1013&ssl=1",1800,1013,true],"chromenews-featured":["https:\/\/i0.wp.com\/ddi.mohflo.net\/wp-content\/uploads\/2025\/01\/owasps-new-llm-top-10-shows-emerging-ai-threats.jpg?fit=1024%2C576&ssl=1",1024,576,true],"chromenews-large":["https:\/\/i0.wp.com\/ddi.mohflo.net\/wp-content\/uploads\/2025\/01\/owasps-new-llm-top-10-shows-emerging-ai-threats.jpg?resize=825%2C575&ssl=1",825,575,true],"chromenews-medium":["https:\/\/i0.wp.com\/ddi.mohflo.net\/wp-content\/uploads\/2025\/01\/owasps-new-llm-top-10-shows-emerging-ai-threats.jpg?resize=590%2C410&ssl=1",590,410,true]},"author_info":{"display_name":"Dark Reading","author_link":"https:\/\/ddi.mohflo.net\/index.php\/author\/darkreading\/"},"category_info":"Uncategorized<\/a>","tag_info":"Uncategorized","comment_count":"0","jetpack_featured_media_url":"https:\/\/i0.wp.com\/ddi.mohflo.net\/wp-content\/uploads\/2025\/01\/owasps-new-llm-top-10-shows-emerging-ai-threats.jpg?fit=1800%2C1013&ssl=1","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/ddi.mohflo.net\/index.php\/wp-json\/wp\/v2\/posts\/6924","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/ddi.mohflo.net\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/ddi.mohflo.net\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/ddi.mohflo.net\/index.php\/wp-json\/wp\/v2\/users\/12"}],"replies":[{"embeddable":true,"href":"https:\/\/ddi.mohflo.net\/index.php\/wp-json\/wp\/v2\/comments?post=6924"}],"version-history":[{"count":0,"href":"https:\/\/ddi.mohflo.net\/index.php\/wp-json\/wp\/v2\/posts\/6924\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/ddi.mohflo.net\/index.php\/wp-json\/wp\/v2\/media\/6925"}],"wp:attachment":[{"href":"https:\/\/ddi.mohflo.net\/index.php\/wp-json\/wp\/v2\/media?parent=6924"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/ddi.mohflo.net\/index.php\/wp-json\/wp\/v2\/categories?post=6924"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/ddi.mohflo.net\/index.php\/wp-json\/wp\/v2\/tags?post=6924"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}