{"id":1938,"date":"2023-10-26T22:00:00","date_gmt":"2023-10-26T22:00:00","guid":{"rendered":"https:\/\/www.darkreading.com\/dr-tech\/iriusrisk-brings-threat-modeling-to-machine-learning"},"modified":"2023-10-26T22:00:00","modified_gmt":"2023-10-26T22:00:00","slug":"iriusrisk-brings-threat-modeling-to-machine-learning-systems","status":"publish","type":"post","link":"https:\/\/ddi.mohflo.net\/index.php\/2023\/10\/26\/iriusrisk-brings-threat-modeling-to-machine-learning-systems\/","title":{"rendered":"IriusRisk Brings Threat Modeling to Machine Learning Systems"},"content":{"rendered":"

<\/p>\n

<\/div>\n

As part of “shift left” to incorporate security discussions earlier in the software development lifecycle, organizations are beginning to look at threat modeling to identify security flaws in software design. With developers increasingly incorporating machine learning in their applications, threat modeling is necessary for identifying the risks to the organization.<\/p>\n

“People are still grappling with the whole idea that when you use that very new technology [machine learning], it brings along a bunch of risk, as well,” says Gary McGraw, co-founder of Berryville Institute of Machine Learning<\/a>. “I’ve been in the unenviable position of saying, \u2018Well, there’s this risk, and there’s that risk, and the sky is falling,\u2019 and everybody goes, \u2018Well, what am I supposed to do about that?”<\/p>\n

There have been a lot of conversations about machine learning risk for quite some time now, but the difficulty lies in figuring out how to address them, McGraw says. Threat modeling \u2013 identifying the types of threats that can cause harm to the organization \u2013 helps organizations think through security risks in machine learning systems<\/a> such as data poisoning, input manipulation, and data extraction. If developers could understand the security flaws in their designs by threat modeling, it would reduce the time spent on security testing during development and before production. The NIST Guidelines on Minimum Standards for Developer Verification of Software<\/a> recommends threat modeling to look for design-level security issues.<\/p>\n

IriusRisk\u2019s threat modeling tool addresses this challenge by automating both threat modeling and architecture risk analysis. Developers and security teams can import the code into the tool to generate diagrams and threat models. Threat modeling templates<\/a> make threat modeling accessible even to those not familiar with diagramming tools or risk analysis.<\/p>\n

And the newly launched AI & ML Security Library allows organizations using IriusRisk to threat model the machine learning system they are planning in order to understand what the security risks are, as well as how to mitigate those risks.<\/p>\n

\u201cWe’re finally getting around to building machinery that people can use to address the risk and control the risk,” says McGraw, who is also a member of IriusRisk\u2019s advisory board. “When you put machine learning into your [system] design, and you\u2019re using IriusRisk, now you know what risks are involved and what to do about that.”<\/p>\n

What ML Threat Modeling Looks Like<\/h2>\n

IriusRisk with the AI & ML Security Library help organizations ask necessary questions. For example:<\/p>\n

    \n
  1. Asking where the data being used to train the machine learning model came from. It’s important to also ask whether anyone had the opportunity to embed incorrect or malicious data to make the machine do the wrong thing.<\/li>\n
  2. Consider how the machine keeps learning once it is in production. Machine learning systems that are online and keep on learning from users are more dangerous than the ones that are not online. “It depends on who is using it. Is it your people? Is it bad people? Is it everybody on Twitter, or X?” McGraw says, noting there have been examples of past projects that had to be taken offline after it learned objectionable information.<\/li>\n
  3. Ask if confidential information can be extracted from the machine. If you put confidential information into your machine learning algorithm, it is not protected by cryptographic means and can be extracted. “If you put the data in the machine, it’s in the machine,” McGraw says. “You need to think about making sure that people using your machine learning system cannot extract that confidential data.”<\/li>\n<\/ol>\n

    The AI & ML Security Library is based on the BIML ML Security Risk Framework, a taxonomy of machine learning threats as well as an architectural risk assessment of typical machine learning components developed by McGraw. The framework is designed to be used by developers, engineers, and designers creating applications and services that use machine learning in the early design and development phases of the project. With IriusRisk’s library, everybody who is using machine learning can now take advantage of BIML’s framework.<\/p>\n

    The AI & ML Security Library is available to both IriusRisk customers and those using the community edition of the platform.<\/p>\n

    Time to be Threat Modeling<\/h2>\n

    The AI & ML Security Library was developed in response to interest from organizations in how to analyze and secure AI and ML systems, according to Stephen de Vries, CEO of IriusRisk. “We have seen a surge in interest from our customers in the finance and technology sectors for guidance on how to analyze, and secure design ML systems,” de Vries said in a statement. “Since these are often new projects that are still in the design phase, performing threat modeling here adds a lot of value, because those teams will very quickly understand where the security goalposts are – and what they need to do in order to get there.”<\/p>\n

    The library doesn’t help organizations who don’t have visibility into their machine learning usage. Just as organizations can have Shadow IT \u2013 where different business stakeholders set up their own servers and web applications without IT oversight \u2013 they can also have shadow machine learning, McGraw says. Different departments are trying out new applications and tools \u2013 but there is a gap between what individual employees are using and what risks IT and security teams know about.<\/p>\n

    \u201cEverybody’s like, \u2018I don’t think I have any machine learning in my organization,\u2019” says McGraw. “But as soon as they find out that they do\u2026 they find it everywhere.\u201d<\/p>\n

    Many organizations do not incorporate threat modeling<\/a> during software design, and those that do rely on manual processes where a person analyzes the threats one at a time.<\/p>\n

    “If you have a mature threat modeling program and you’re using a tool like IriusRisk, you can also now handle machine learning. So the people that are already doing the best are going to do even better,” McGraw says. “What about the people who aren’t doing threat modeling? Maybe they should start. It’s not new. It’s time to do it.”<\/p>\n

    Source<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"

    As part of “shift left” to incorporate security discussions earlier<\/p>\n","protected":false},"author":12,"featured_media":1939,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[1],"tags":[809],"class_list":["post-1938","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-uncategorized","tag-dark-reading"],"featured_image_urls":{"full":["https:\/\/i0.wp.com\/ddi.mohflo.net\/wp-content\/uploads\/2023\/10\/iriusrisk-brings-threat-modeling-to-machine-learning-systems.png?fit=125%2C125&ssl=1",125,125,false],"thumbnail":["https:\/\/i0.wp.com\/ddi.mohflo.net\/wp-content\/uploads\/2023\/10\/iriusrisk-brings-threat-modeling-to-machine-learning-systems.png?resize=125%2C125&ssl=1",125,125,true],"medium":["https:\/\/i0.wp.com\/ddi.mohflo.net\/wp-content\/uploads\/2023\/10\/iriusrisk-brings-threat-modeling-to-machine-learning-systems.png?fit=125%2C125&ssl=1",125,125,true],"medium_large":["https:\/\/i0.wp.com\/ddi.mohflo.net\/wp-content\/uploads\/2023\/10\/iriusrisk-brings-threat-modeling-to-machine-learning-systems.png?fit=125%2C125&ssl=1",125,125,true],"large":["https:\/\/i0.wp.com\/ddi.mohflo.net\/wp-content\/uploads\/2023\/10\/iriusrisk-brings-threat-modeling-to-machine-learning-systems.png?fit=125%2C125&ssl=1",125,125,true],"1536x1536":["https:\/\/i0.wp.com\/ddi.mohflo.net\/wp-content\/uploads\/2023\/10\/iriusrisk-brings-threat-modeling-to-machine-learning-systems.png?fit=125%2C125&ssl=1",125,125,true],"2048x2048":["https:\/\/i0.wp.com\/ddi.mohflo.net\/wp-content\/uploads\/2023\/10\/iriusrisk-brings-threat-modeling-to-machine-learning-systems.png?fit=125%2C125&ssl=1",125,125,true],"chromenews-featured":["https:\/\/i0.wp.com\/ddi.mohflo.net\/wp-content\/uploads\/2023\/10\/iriusrisk-brings-threat-modeling-to-machine-learning-systems.png?fit=125%2C125&ssl=1",125,125,true],"chromenews-large":["https:\/\/i0.wp.com\/ddi.mohflo.net\/wp-content\/uploads\/2023\/10\/iriusrisk-brings-threat-modeling-to-machine-learning-systems.png?resize=125%2C125&ssl=1",125,125,true],"chromenews-medium":["https:\/\/i0.wp.com\/ddi.mohflo.net\/wp-content\/uploads\/2023\/10\/iriusrisk-brings-threat-modeling-to-machine-learning-systems.png?resize=125%2C125&ssl=1",125,125,true]},"author_info":{"display_name":"Dark Reading","author_link":"https:\/\/ddi.mohflo.net\/index.php\/author\/darkreading\/"},"category_info":"Uncategorized<\/a>","tag_info":"Uncategorized","comment_count":"0","jetpack_featured_media_url":"https:\/\/i0.wp.com\/ddi.mohflo.net\/wp-content\/uploads\/2023\/10\/iriusrisk-brings-threat-modeling-to-machine-learning-systems.png?fit=125%2C125&ssl=1","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/ddi.mohflo.net\/index.php\/wp-json\/wp\/v2\/posts\/1938","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/ddi.mohflo.net\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/ddi.mohflo.net\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/ddi.mohflo.net\/index.php\/wp-json\/wp\/v2\/users\/12"}],"replies":[{"embeddable":true,"href":"https:\/\/ddi.mohflo.net\/index.php\/wp-json\/wp\/v2\/comments?post=1938"}],"version-history":[{"count":0,"href":"https:\/\/ddi.mohflo.net\/index.php\/wp-json\/wp\/v2\/posts\/1938\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/ddi.mohflo.net\/index.php\/wp-json\/wp\/v2\/media\/1939"}],"wp:attachment":[{"href":"https:\/\/ddi.mohflo.net\/index.php\/wp-json\/wp\/v2\/media?parent=1938"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/ddi.mohflo.net\/index.php\/wp-json\/wp\/v2\/categories?post=1938"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/ddi.mohflo.net\/index.php\/wp-json\/wp\/v2\/tags?post=1938"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}