{"id":2956,"date":"2024-04-05T12:00:00","date_gmt":"2024-04-05T17:00:00","guid":{"rendered":"https:\/\/www.darkreading.com\/application-security\/how-do-we-integrate-llm-security-into-application-development-"},"modified":"2024-04-05T12:00:00","modified_gmt":"2024-04-05T17:00:00","slug":"how-do-we-integrate-llms-security-into-application-development","status":"publish","type":"post","link":"https:\/\/ddi.mohflo.net\/index.php\/2024\/04\/05\/how-do-we-integrate-llms-security-into-application-development\/","title":{"rendered":"How Do We Integrate LLMs Security Into Application Development?"},"content":{"rendered":"<div class=\"media_block\"><a href=\"https:\/\/i0.wp.com\/eu-images.contentstack.com\/v3\/assets\/blt6d90778a997de1cd\/blt6781e27cfba8a796\/65f0b3517410c5040aecc611\/aiappsec-Chroma_Craft_Media_Group-alamy.png?ssl=1\"><img data-recalc-dims=\"1\" decoding=\"async\" src=\"https:\/\/i0.wp.com\/ddi.mohflo.net\/wp-content\/uploads\/2024\/04\/how-do-we-integrate-llms-security-into-application-development.png?w=640&#038;ssl=1\" class=\"media_thumbnail\"><\/a><\/div>\n<div><img data-recalc-dims=\"1\" decoding=\"async\" src=\"https:\/\/i0.wp.com\/ddi.mohflo.net\/wp-content\/uploads\/2024\/04\/how-do-we-integrate-llms-security-into-application-development.png?w=640&#038;ssl=1\" class=\"ff-og-image-inserted\"><\/div>\n<p class=\"ContentParagraph ContentParagraph_align_left\" data-testid=\"content-paragraph\"><span class=\"ContentText ContentText_variant_bodyNormal\" data-testid=\"content-text\"><span class=\"ContentText-BodyTextChunk ContentText-BodyTextChunk_bold\">Question: What do we really know about large language model (LLM) security? And are we willingly opening the front door to chaos by using LLMs in business?<\/span><\/span><\/p>\n<p class=\"ContentParagraph ContentParagraph_align_left\" data-testid=\"content-paragraph\"><span class=\"ContentText ContentText_variant_bodyNormal\" data-testid=\"content-text\"><span class=\"ContentText-BodyTextChunk ContentText-BodyTextChunk_bold\">Rob Gurzeev, CEO, CyCognito:<\/span><\/span><span class=\"ContentText ContentText_variant_bodyNormal\" data-testid=\"content-text\"> Picture it: Your engineering team is harnessing the immense capabilities of LLMs to &#8220;write code&#8221; and rapidly develop an application. It&#8217;s a game-changer for your businesses; development speeds are now orders of magnitude faster. You&#8217;ve shaved 30% off time-to-market. It&#8217;s win-win \u2014 for your org, your stakeholders, your end users.<\/span><\/p>\n<p class=\"ContentParagraph ContentParagraph_align_left\" data-testid=\"content-paragraph\"><span class=\"ContentText ContentText_variant_bodyNormal\" data-testid=\"content-text\">Six months later, your application is reported to leak customer data; it has been jailbroken and its code manipulated. You&#8217;re now <\/span><span class=\"ContentText ContentText_variant_bodyNormal\" data-testid=\"content-text\"><a href=\"https:\/\/www.darkreading.com\/application-security\/solarwinds-faces-potential-sec-enforcement-act-over-orion-breach\" target=\"_blank\" class=\"ContentText-BodyTextChunk ContentText-BodyTextChunk_link\" rel=\"noopener\">facing SEC violations<\/a><\/span><span class=\"ContentText ContentText_variant_bodyNormal\" data-testid=\"content-text\"> and the threat of customers walking away.<\/span><\/p>\n<p class=\"ContentParagraph ContentParagraph_align_left\" data-testid=\"content-paragraph\"><span class=\"ContentText ContentText_variant_bodyNormal\" data-testid=\"content-text\">Efficiency gains are enticing, but the risks cannot be ignored. While we have well-established standards for security in traditional software development, LLMs are black boxes that require rethinking how we bake in security.<\/span><\/p>\n<h2 class=\"ContentText ContentText_variant_h2 ContentText_align_left\" data-testid=\"content-text\" id=\"New Kinds of Security Risks for LLMs\">New Kinds of Security Risks for LLMs<\/h2>\n<p class=\"ContentParagraph ContentParagraph_align_left\" data-testid=\"content-paragraph\"><span class=\"ContentText ContentText_variant_bodyNormal\" data-testid=\"content-text\">LLMs are rife with unknown risks and prone to attacks previously unseen in traditional software development.<\/span><\/p>\n<div data-component=\"basic-list\" class=\"BasicList BasicList_nestedLevel_0 BasicList_variant_unordered BasicList_limited\">\n<ul data-testid=\"basic-list-unordered\" class=\"BasicList-UnorderedList\">\n<li>\n<div class=\"BasicList-ListItem BasicList-ListItem_variant_unordered\" readability=\"9.723\"><span data-component=\"icon\" data-name=\"Circle\" class=\"BasicList-ListIcon BasicList-ListIcon_variant_unordered\"><\/span><\/p>\n<div class=\"BasicList-Item\" readability=\"14.816\">\n<p class=\"ContentParagraph ContentParagraph_align_left\" data-testid=\"content-paragraph\"><span class=\"ContentText ContentText_variant_bodyNormal\" data-testid=\"content-text\"><span class=\"ContentText-BodyTextChunk ContentText-BodyTextChunk_bold\">Prompt injection attacks<\/span><\/span><span class=\"ContentText ContentText_variant_bodyNormal\" data-testid=\"content-text\"> involve manipulating the model to generate unintended or harmful responses. Here, the attacker strategically <\/span><span class=\"ContentText ContentText_variant_bodyNormal\" data-testid=\"content-text\"><a href=\"https:\/\/www.darkreading.com\/cyber-risk\/forget-deepfakes-or-phishing-prompt-injection-is-genai-s-biggest-problem\" target=\"_blank\" class=\"ContentText-BodyTextChunk ContentText-BodyTextChunk_link\" rel=\"noopener\">formulates prompts to deceive the LLM<\/a><\/span><span class=\"ContentText ContentText_variant_bodyNormal\" data-testid=\"content-text\">, potentially bypassing security measures or ethical constraints put in place to ensure responsible use of the artificial intelligence (AI). As a result, the LLM&#8217;s responses can deviate significantly from the intended or expected behavior, posing serious risks to privacy, security, and the reliability of AI-driven applications.<\/span><\/p>\n<\/div>\n<\/div>\n<\/li>\n<li>\n<div class=\"BasicList-ListItem BasicList-ListItem_variant_unordered\" readability=\"9.1803278688525\"><span data-component=\"icon\" data-name=\"Circle\" class=\"BasicList-ListIcon BasicList-ListIcon_variant_unordered\"><\/span><\/p>\n<div class=\"BasicList-Item\" readability=\"13.770491803279\">\n<p class=\"ContentParagraph ContentParagraph_align_left\" data-testid=\"content-paragraph\"><span class=\"ContentText ContentText_variant_bodyNormal\" data-testid=\"content-text\"><span class=\"ContentText-BodyTextChunk ContentText-BodyTextChunk_bold\">Insecure output handling<\/span><\/span><span class=\"ContentText ContentText_variant_bodyNormal\" data-testid=\"content-text\"> arises when the output generated by an LLM or similar AI system is accepted and incorporated into a software application or Web service without undergoing adequate scrutiny or validation. This can expose <\/span><span class=\"ContentText ContentText_variant_bodyNormal\" data-testid=\"content-text\"><a href=\"https:\/\/www.darkreading.com\/application-security\/coding-tips-to-sidestep-javascript-vulnerabilities\" target=\"_blank\" class=\"ContentText-BodyTextChunk ContentText-BodyTextChunk_link\" rel=\"noopener\">back-end systems to vulnerabilities<\/a><\/span><span class=\"ContentText ContentText_variant_bodyNormal\" data-testid=\"content-text\">, such as cross-site scripting (XSS), cross-site request forgery (CSRF), server-side request forgery (SSRF), privilege escalation, and remote code execution (RCE).<\/span><\/p>\n<\/div>\n<\/div>\n<\/li>\n<li>\n<div class=\"BasicList-ListItem BasicList-ListItem_variant_unordered\" readability=\"8.5\"><span data-component=\"icon\" data-name=\"Circle\" class=\"BasicList-ListIcon BasicList-ListIcon_variant_unordered\"><\/span><\/p>\n<div class=\"BasicList-Item\" readability=\"12\">\n<p class=\"ContentParagraph ContentParagraph_align_left\" data-testid=\"content-paragraph\"><span class=\"ContentText ContentText_variant_bodyNormal\" data-testid=\"content-text\"><span class=\"ContentText-BodyTextChunk ContentText-BodyTextChunk_bold\">Training data poisoning<\/span><\/span><span class=\"ContentText ContentText_variant_bodyNormal\" data-testid=\"content-text\"> occurs when the data used to train an LLM is deliberately manipulated or contaminated with malicious or biased information. The process of training data poisoning typically involves the injection of deceptive, misleading, or harmful data points into the training dataset. These manipulated data instances are strategically chosen to exploit vulnerabilities in the model&#8217;s learning algorithms or to instill biases that may lead to undesired outcomes in the model&#8217;s predictions and responses.<\/span><\/p>\n<\/div>\n<\/div>\n<\/li>\n<\/ul>\n<\/div>\n<h2 class=\"ContentText ContentText_variant_h2 ContentText_align_left\" data-testid=\"content-text\" id=\"A Blueprint for Protection and Control of LLM Applications\">A Blueprint for Protection and Control of LLM Applications<\/h2>\n<p class=\"ContentParagraph ContentParagraph_align_left\" data-testid=\"content-paragraph\"><span class=\"ContentText ContentText_variant_bodyNormal\" data-testid=\"content-text\">While some of this is <\/span><span class=\"ContentText ContentText_variant_bodyNormal\" data-testid=\"content-text\"><a href=\"https:\/\/www.darkreading.com\/application-security\/security-must-empower-ai-developers-now\" target=\"_blank\" class=\"ContentText-BodyTextChunk ContentText-BodyTextChunk_link\" rel=\"noopener\">new territory<\/a><\/span><span class=\"ContentText ContentText_variant_bodyNormal\" data-testid=\"content-text\">, there are best practices you can implement to limit exposure.<\/span><\/p>\n<div data-component=\"basic-list\" class=\"BasicList BasicList_nestedLevel_0 BasicList_variant_unordered BasicList_limited\">\n<ul data-testid=\"basic-list-unordered\" class=\"BasicList-UnorderedList\">\n<li>\n<div class=\"BasicList-ListItem BasicList-ListItem_variant_unordered\" readability=\"12.5\"><span data-component=\"icon\" data-name=\"Circle\" class=\"BasicList-ListIcon BasicList-ListIcon_variant_unordered\"><\/span><\/p>\n<div class=\"BasicList-Item\" readability=\"20\">\n<p class=\"ContentParagraph ContentParagraph_align_left\" data-testid=\"content-paragraph\"><span class=\"ContentText ContentText_variant_bodyNormal\" data-testid=\"content-text\"><span class=\"ContentText-BodyTextChunk ContentText-BodyTextChunk_bold\">Input sanitization<\/span><\/span><span class=\"ContentText ContentText_variant_bodyNormal\" data-testid=\"content-text\"> involves, as the name suggestion, the<\/span><span class=\"ContentText ContentText_variant_bodyNormal\" data-testid=\"content-text\"><span class=\"ContentText-BodyTextChunk ContentText-BodyTextChunk_italic\"> <\/span><\/span><span class=\"ContentText ContentText_variant_bodyNormal\" data-testid=\"content-text\">sanitization of inputs to prevent unauthorized actions and data requests initiated by malicious prompts. The first step is input validation to ensure input adheres to expected formats and data types. The next is input sanitization, where potentially harmful characters or code are removed or encoded to thwart attacks. Other tactics include whitelists of approved content, blacklists of forbidden content, parameterized queries for database interactions, content security policies, regular expressions, logging, and continuous monitoring, as well as security updates and testing.<\/span><\/p>\n<\/div>\n<\/div>\n<\/li>\n<li>\n<div class=\"BasicList-ListItem BasicList-ListItem_variant_unordered\" readability=\"10.5\"><span data-component=\"icon\" data-name=\"Circle\" class=\"BasicList-ListIcon BasicList-ListIcon_variant_unordered\"><\/span><\/p>\n<div class=\"BasicList-Item\" readability=\"16\">\n<p class=\"ContentParagraph ContentParagraph_align_left\" data-testid=\"content-paragraph\"><span class=\"ContentText ContentText_variant_bodyNormal\" data-testid=\"content-text\"><span class=\"ContentText-BodyTextChunk ContentText-BodyTextChunk_bold\">Output scrutiny <\/span><\/span><span class=\"ContentText ContentText_variant_bodyNormal\" data-testid=\"content-text\">is<\/span><span class=\"ContentText ContentText_variant_bodyNormal\" data-testid=\"content-text\"><span class=\"ContentText-BodyTextChunk ContentText-BodyTextChunk_bold\"> <\/span><\/span><span class=\"ContentText ContentText_variant_bodyNormal\" data-testid=\"content-text\">the rigorous handling and evaluation of the output generated by the LLM to mitigate vulnerabilities, like XSS, CSRF, and RCE. The process begins by validating and filtering the LLM&#8217;s responses before accepting them for presentation or further processing. It incorporates techniques such as content validation, output encoding, and output escaping, all of which aim to identify and neutralize potential security risks in the generated content.<\/span><\/p>\n<\/div>\n<\/div>\n<\/li>\n<li>\n<div class=\"BasicList-ListItem BasicList-ListItem_variant_unordered\" readability=\"10.5\"><span data-component=\"icon\" data-name=\"Circle\" class=\"BasicList-ListIcon BasicList-ListIcon_variant_unordered\"><\/span><\/p>\n<div class=\"BasicList-Item\" readability=\"16\">\n<p class=\"ContentParagraph ContentParagraph_align_left\" data-testid=\"content-paragraph\"><span class=\"ContentText ContentText_variant_bodyNormal\" data-testid=\"content-text\"><span class=\"ContentText-BodyTextChunk ContentText-BodyTextChunk_bold\">Safeguarding training data<\/span><\/span><span class=\"ContentText ContentText_variant_bodyNormal\" data-testid=\"content-text\"> is essential to prevent training data poisoning. This involves enforcing strict access controls, employing encryption for data protection, maintaining data backups and version control, implementing data validation and anonymization, establishing comprehensive logging and monitoring, conducting regular audits, and providing employee training on data security. It&#8217;s also important to verify the reliability of data sources and ensure secure storage and transmission practices.<\/span><\/p>\n<\/div>\n<\/div>\n<\/li>\n<li>\n<div class=\"BasicList-ListItem BasicList-ListItem_variant_unordered\" readability=\"11.5\"><span data-component=\"icon\" data-name=\"Circle\" class=\"BasicList-ListIcon BasicList-ListIcon_variant_unordered\"><\/span><\/p>\n<div class=\"BasicList-Item\" readability=\"18\">\n<p class=\"ContentParagraph ContentParagraph_align_left\" data-testid=\"content-paragraph\"><span class=\"ContentText ContentText_variant_bodyNormal\" data-testid=\"content-text\"><span class=\"ContentText-BodyTextChunk ContentText-BodyTextChunk_bold\">Enforcing strict sandboxing policies and access controls<\/span><\/span><span class=\"ContentText ContentText_variant_bodyNormal\" data-testid=\"content-text\"> can also help mitigate the risk of SSRF exploits in LLM operations. Techniques that can be applied here include sandbox isolation, access controls, whitelisting and\/or blacklisting, request validation, network segmentation, content-type validation, and content inspection. Regular updates, comprehensive logging, and employee training are also key.<\/span><\/p>\n<\/div>\n<\/div>\n<\/li>\n<li>\n<div class=\"BasicList-ListItem BasicList-ListItem_variant_unordered\" readability=\"11\"><span data-component=\"icon\" data-name=\"Circle\" class=\"BasicList-ListIcon BasicList-ListIcon_variant_unordered\"><\/span><\/p>\n<div class=\"BasicList-Item\" readability=\"17\">\n<p class=\"ContentParagraph ContentParagraph_align_left\" data-testid=\"content-paragraph\"><span class=\"ContentText ContentText_variant_bodyNormal\" data-testid=\"content-text\"><span class=\"ContentText-BodyTextChunk ContentText-BodyTextChunk_bold\">Continuous monitoring and content filtering<\/span><\/span><span class=\"ContentText ContentText_variant_bodyNormal\" data-testid=\"content-text\"> can be integrated into the LLM&#8217;s processing pipeline to detect and prevent harmful or inappropriate content, using keyword-based filtering, contextual analysis, machine-learning models, and customizable filters. Ethical guidelines and human moderation play key roles in maintaining responsible content generation, while continuous real-time monitoring, user feedback loops, and transparency ensure that any deviations from desired behavior are promptly addressed.<\/span><\/p>\n<\/div>\n<\/div>\n<\/li>\n<\/ul>\n<\/div>\n<p><a href=\"https:\/\/www.darkreading.com\/application-security\/how-do-we-integrate-llm-security-into-application-development-\">Source<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Question: What do we really know about large language model<\/p>\n","protected":false},"author":12,"featured_media":2957,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[1],"tags":[809],"class_list":["post-2956","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-uncategorized","tag-dark-reading"],"featured_image_urls":{"full":["https:\/\/i0.wp.com\/ddi.mohflo.net\/wp-content\/uploads\/2024\/04\/how-do-we-integrate-llms-security-into-application-development.png?fit=1800%2C1012&ssl=1",1800,1012,false],"thumbnail":["https:\/\/i0.wp.com\/ddi.mohflo.net\/wp-content\/uploads\/2024\/04\/how-do-we-integrate-llms-security-into-application-development.png?resize=150%2C150&ssl=1",150,150,true],"medium":["https:\/\/i0.wp.com\/ddi.mohflo.net\/wp-content\/uploads\/2024\/04\/how-do-we-integrate-llms-security-into-application-development.png?fit=300%2C169&ssl=1",300,169,true],"medium_large":["https:\/\/i0.wp.com\/ddi.mohflo.net\/wp-content\/uploads\/2024\/04\/how-do-we-integrate-llms-security-into-application-development.png?fit=640%2C360&ssl=1",640,360,true],"large":["https:\/\/i0.wp.com\/ddi.mohflo.net\/wp-content\/uploads\/2024\/04\/how-do-we-integrate-llms-security-into-application-development.png?fit=640%2C360&ssl=1",640,360,true],"1536x1536":["https:\/\/i0.wp.com\/ddi.mohflo.net\/wp-content\/uploads\/2024\/04\/how-do-we-integrate-llms-security-into-application-development.png?fit=1536%2C864&ssl=1",1536,864,true],"2048x2048":["https:\/\/i0.wp.com\/ddi.mohflo.net\/wp-content\/uploads\/2024\/04\/how-do-we-integrate-llms-security-into-application-development.png?fit=1800%2C1012&ssl=1",1800,1012,true],"chromenews-featured":["https:\/\/i0.wp.com\/ddi.mohflo.net\/wp-content\/uploads\/2024\/04\/how-do-we-integrate-llms-security-into-application-development.png?fit=1024%2C576&ssl=1",1024,576,true],"chromenews-large":["https:\/\/i0.wp.com\/ddi.mohflo.net\/wp-content\/uploads\/2024\/04\/how-do-we-integrate-llms-security-into-application-development.png?resize=825%2C575&ssl=1",825,575,true],"chromenews-medium":["https:\/\/i0.wp.com\/ddi.mohflo.net\/wp-content\/uploads\/2024\/04\/how-do-we-integrate-llms-security-into-application-development.png?resize=590%2C410&ssl=1",590,410,true]},"author_info":{"display_name":"Dark Reading","author_link":"https:\/\/ddi.mohflo.net\/index.php\/author\/darkreading\/"},"category_info":"<a href=\"https:\/\/ddi.mohflo.net\/index.php\/category\/uncategorized\/\" rel=\"category tag\">Uncategorized<\/a>","tag_info":"Uncategorized","comment_count":"0","jetpack_featured_media_url":"https:\/\/i0.wp.com\/ddi.mohflo.net\/wp-content\/uploads\/2024\/04\/how-do-we-integrate-llms-security-into-application-development.png?fit=1800%2C1012&ssl=1","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/ddi.mohflo.net\/index.php\/wp-json\/wp\/v2\/posts\/2956","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/ddi.mohflo.net\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/ddi.mohflo.net\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/ddi.mohflo.net\/index.php\/wp-json\/wp\/v2\/users\/12"}],"replies":[{"embeddable":true,"href":"https:\/\/ddi.mohflo.net\/index.php\/wp-json\/wp\/v2\/comments?post=2956"}],"version-history":[{"count":0,"href":"https:\/\/ddi.mohflo.net\/index.php\/wp-json\/wp\/v2\/posts\/2956\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/ddi.mohflo.net\/index.php\/wp-json\/wp\/v2\/media\/2957"}],"wp:attachment":[{"href":"https:\/\/ddi.mohflo.net\/index.php\/wp-json\/wp\/v2\/media?parent=2956"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/ddi.mohflo.net\/index.php\/wp-json\/wp\/v2\/categories?post=2956"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/ddi.mohflo.net\/index.php\/wp-json\/wp\/v2\/tags?post=2956"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}