{"id":6635,"date":"2024-12-16T14:33:16","date_gmt":"2024-12-16T20:33:16","guid":{"rendered":"https:\/\/www.darkreading.com\/application-security\/does-desktop-ai-risk"},"modified":"2024-12-16T14:33:16","modified_gmt":"2024-12-16T20:33:16","slug":"does-desktop-ai-come-with-a-side-of-risk","status":"publish","type":"post","link":"https:\/\/ddi.mohflo.net\/index.php\/2024\/12\/16\/does-desktop-ai-come-with-a-side-of-risk\/","title":{"rendered":"Does Desktop AI Come With a Side of Risk?"},"content":{"rendered":"
<\/a><\/div>\n
<\/div>\n

Artificial intelligence has come to the desktop.<\/span><\/p>\n

Microsoft 365 Copilot, which debuted last year, is now widely available. Apple Intelligence just reached general beta availability for users of late-model Macs, iPhones, and iPads. And Google Gemini will reportedly soon be able to take actions through the Chrome browser under an in-development agent feature dubbed Project Jarvis.<\/span><\/p>\n

The integration of large language models (LLMs) that sift through business information and provide automated scripting of actions \u2014 so-called “agentic” capabilities \u2014 holds massive promise for knowledge workers but also significant concerns for business leaders and chief information security officers (CISOs). Companies already suffer from significant issues with the oversharing of information and a failure to limit access permissions \u2014 40% of firms delayed their rollout of Microsoft 365 Copilot by three months or more because of such security worries, according to a <\/span>Gartner survey<\/a><\/span>.<\/span><\/p>\n

The broad range of capabilities offered by desktop AI systems, combined with the lack of rigorous information security at many businesses, poses a significant risk, says Jim Alkove, CEO of Oleria, an identity and access management platform for cloud services.<\/span><\/p>\n

“It’s the combinatorics here that actually should make everyone concerned,” he says. “These categorical risks exist in the larger [native language] model-based technology, and when you combine them with the sort of runtime security risks that we’ve been dealing with \u2014 and information access and auditability risks \u2014 it ends up having a multiplicative effect on risk.”<\/span><\/p>\n

Related:<\/span>Citizen Development Moves Too Fast for Its Own Good<\/a><\/p>\n

Desktop AI will likely take off in 2025. Companies are already looking to rapidly adopt Microsoft 365 Copilot and other desktop AI technologies, but only 16% have pushed past initial pilot projects to roll out the technology to all workers, according to <\/span>Gartner’s “The State of Microsoft 365 Copilot: Survey Results.”<\/a><\/span> The overwhelming majority (60%) are still evaluating the technology in a pilot project, while a fifth of businesses haven’t even reached that far and are still in the planning stage.<\/span><\/p>\n

Most workers are looking forward to having a desktop AI system to assist them with daily tasks. Some 90% of respondents believe their users would fight to retain access to their AI assistant, and 89% agree that the technology has improved productivity, according to Gartner.<\/span><\/p>\n

Bringing Security to the AI Assistant<\/h2>\n

Unfortunately, the technologies are black boxes in terms of their architecture and protections, and that means they lack trust. With a human personal assistant, companies can do background checks, limit their access to certain technologies, and audit their work \u2014 measures that have no analogous control with desktop AI systems at present, says Oleria’s Alkove.<\/span><\/p>\n

Related:<\/span>Cleo MFT Zero-Day Exploits Are About to Escalate, Analysts Warn<\/a><\/p>\n

AI assistants \u2014 whether they are on the desktop, on a mobile device, or in the cloud \u2014 will have far more access to information than they need, he says.<\/span><\/p>\n

“If you think about how ill-equipped modern technology is to deal with the fact that my assistant should be able to do a certain set of electronic tasks on my behalf, but nothing else,” Alkove says. “You can grant your assistant access to email and your calendar, but you cannot restrict your assistant from seeing certain emails and certain calendar events. They can see everything.”<\/span><\/p>\n

This ability to delegate tasks needs to become part of the security fabric of AI assistants, he says.<\/span><\/p>\n

Cyber-Risk: Social Engineering Both Users & AI<\/h2>\n

Without such security design and controls, attacks will likely follow.<\/span><\/p>\n

Earlier this year, a prompt injection attack scenario highlighted the risks to businesses. Security researcher Johann Rehberger found that an indirect prompt injection attack through email, a Word document, or a website <\/span>could trick Microsoft 365 Copilot into taking on the role of a scammer<\/a><\/span>, extracting personal information, and leaking it to an attacker. Rehberger initially notified Microsoft of the issue in January and provided the company with information throughout the year. It’s unknown whether Microsoft has a comprehensive fix for the issue.<\/span><\/p>\n

Related:<\/span>Generative AI Security Tools Go Open Source<\/a><\/p>\n

The ability to access the capabilities of an operating system or device will make desktop AI assistants another target for fraudsters who have been trying to get a user to take actions. Instead, they will now focus on getting an LLM to take actions, says Ben Kilger, CEO of Zenity, an AI agent security firm.<\/span><\/p>\n

“An LLM gives them the ability to do things on your behalf without any specific consent or control,” he says. “So many of these prompt injection attacks are trying to social engineer the system \u2014 trying to go around other controls that you have in your network without having to socially engineer a human.”<\/span><\/p>\n

Visibility Into AI’s Black Box<\/h2>\n

Most companies lack visibility into and control of the security of AI technology in general. To adequately vet the technology, companies need to be able to examine what the AI system is doing, how employees are interacting with the technology, and what actions are being delegated to the AI, Kilger says.<\/span><\/p>\n

“These are all things that the organization needs to control, not the agentic platform,” he says. “You need to break it down and to actually look deeper into how those platforms actually being utilized, and how do people build and interact with those platforms.”<\/span><\/p>\n

The first step to evaluating the risk of Microsoft 365 Copilot, Google’s purported Project Jarvis, Apple Intelligence, and other technologies is to gain this visibility and have the controls in place to limit an AI assistant’s access on a granular level, says Oleria’s Alkove.<\/span><\/p>\n

Rather than a big bucket of data that a desktop AI system can always access, companies need to be able to control access by the eventual recipient of the data, their role, and the sensitivity of the information, he says.<\/span><\/p>\n

“How do you grant access to portions of your information and portions of the actions that you would normally take as an individual, to that agent, and also only for a period of time?” Alkove asks. “You might only want the agent to take an action once, or you may only want them to do it for 24 hours, and so making sure that you have those kind of controls today is critical.”<\/span><\/p>\n

Microsoft, for its part, acknowledges the data-governance challenges, but argues that they are not new, just made more apparent due to AI\u2019s arrival.<\/span><\/p>\n

“AI is simply the latest call to action for enterprises to take proactive management of controls their unique, respective policies, industry compliance regulations, and risk tolerance should inform \u2013 such as determining which employee identities should have access to different types of files, workspaces, and other resources,” a company spokesperson said in a statement.<\/span><\/p>\n

The company pointed to its Microsoft Purview portal as a way that organizations can continuously manage identities, permission, and other controls. Using the portal, IT admins can help secure data for AI apps and proactively monitor AI use though a single management location, the company said. Google declined to comment about its forthcoming AI agent.<\/span><\/p>\n

Source<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"

Artificial intelligence has come to the desktop. Microsoft 365 Copilot,<\/p>\n","protected":false},"author":12,"featured_media":6636,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[1],"tags":[809],"class_list":["post-6635","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-uncategorized","tag-dark-reading"],"featured_image_urls":{"full":["https:\/\/i0.wp.com\/ddi.mohflo.net\/wp-content\/uploads\/2024\/12\/does-desktop-ai-come-with-a-side-of-risk.jpg?fit=1920%2C1080&ssl=1",1920,1080,false],"thumbnail":["https:\/\/i0.wp.com\/ddi.mohflo.net\/wp-content\/uploads\/2024\/12\/does-desktop-ai-come-with-a-side-of-risk.jpg?resize=150%2C150&ssl=1",150,150,true],"medium":["https:\/\/i0.wp.com\/ddi.mohflo.net\/wp-content\/uploads\/2024\/12\/does-desktop-ai-come-with-a-side-of-risk.jpg?fit=300%2C169&ssl=1",300,169,true],"medium_large":["https:\/\/i0.wp.com\/ddi.mohflo.net\/wp-content\/uploads\/2024\/12\/does-desktop-ai-come-with-a-side-of-risk.jpg?fit=640%2C360&ssl=1",640,360,true],"large":["https:\/\/i0.wp.com\/ddi.mohflo.net\/wp-content\/uploads\/2024\/12\/does-desktop-ai-come-with-a-side-of-risk.jpg?fit=640%2C360&ssl=1",640,360,true],"1536x1536":["https:\/\/i0.wp.com\/ddi.mohflo.net\/wp-content\/uploads\/2024\/12\/does-desktop-ai-come-with-a-side-of-risk.jpg?fit=1536%2C864&ssl=1",1536,864,true],"2048x2048":["https:\/\/i0.wp.com\/ddi.mohflo.net\/wp-content\/uploads\/2024\/12\/does-desktop-ai-come-with-a-side-of-risk.jpg?fit=1920%2C1080&ssl=1",1920,1080,true],"chromenews-featured":["https:\/\/i0.wp.com\/ddi.mohflo.net\/wp-content\/uploads\/2024\/12\/does-desktop-ai-come-with-a-side-of-risk.jpg?fit=1024%2C576&ssl=1",1024,576,true],"chromenews-large":["https:\/\/i0.wp.com\/ddi.mohflo.net\/wp-content\/uploads\/2024\/12\/does-desktop-ai-come-with-a-side-of-risk.jpg?resize=825%2C575&ssl=1",825,575,true],"chromenews-medium":["https:\/\/i0.wp.com\/ddi.mohflo.net\/wp-content\/uploads\/2024\/12\/does-desktop-ai-come-with-a-side-of-risk.jpg?resize=590%2C410&ssl=1",590,410,true]},"author_info":{"display_name":"Dark Reading","author_link":"https:\/\/ddi.mohflo.net\/index.php\/author\/darkreading\/"},"category_info":"Uncategorized<\/a>","tag_info":"Uncategorized","comment_count":"0","jetpack_featured_media_url":"https:\/\/i0.wp.com\/ddi.mohflo.net\/wp-content\/uploads\/2024\/12\/does-desktop-ai-come-with-a-side-of-risk.jpg?fit=1920%2C1080&ssl=1","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/ddi.mohflo.net\/index.php\/wp-json\/wp\/v2\/posts\/6635","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/ddi.mohflo.net\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/ddi.mohflo.net\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/ddi.mohflo.net\/index.php\/wp-json\/wp\/v2\/users\/12"}],"replies":[{"embeddable":true,"href":"https:\/\/ddi.mohflo.net\/index.php\/wp-json\/wp\/v2\/comments?post=6635"}],"version-history":[{"count":0,"href":"https:\/\/ddi.mohflo.net\/index.php\/wp-json\/wp\/v2\/posts\/6635\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/ddi.mohflo.net\/index.php\/wp-json\/wp\/v2\/media\/6636"}],"wp:attachment":[{"href":"https:\/\/ddi.mohflo.net\/index.php\/wp-json\/wp\/v2\/media?parent=6635"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/ddi.mohflo.net\/index.php\/wp-json\/wp\/v2\/categories?post=6635"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/ddi.mohflo.net\/index.php\/wp-json\/wp\/v2\/tags?post=6635"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}