{"id":6075,"date":"2024-11-04T09:46:28","date_gmt":"2024-11-04T15:46:28","guid":{"rendered":"https:\/\/www.darkreading.com\/application-security\/google-big-sleep-ai-agent-sqlite-software-bug"},"modified":"2024-11-04T09:46:28","modified_gmt":"2024-11-04T15:46:28","slug":"google-big-sleep-ai-agent-puts-sqlite-software-bug-to-bed","status":"publish","type":"post","link":"https:\/\/ddi.mohflo.net\/index.php\/2024\/11\/04\/google-big-sleep-ai-agent-puts-sqlite-software-bug-to-bed\/","title":{"rendered":"Google: Big Sleep AI Agent Puts SQLite Software Bug to Bed"},"content":{"rendered":"
<\/a><\/div>\n
<\/div>\n

Google has discovered its first real-world vulnerability using an artificial intelligence (AI) agent that company researchers are designing expressly for this purpose. The discovery of a memory-safety flaw in a production version of a popular open source database by the company’s Big Sleep large language model (LLM) project is the first of its kind, and it has “tremendous defensive potential” for organizations, the <\/span>Big Sleep team wrote<\/a><\/span> in a recent Project Zero blog.<\/span><\/p>\n

Big Sleep \u2014 the work of a collaboration between the company’s Project Zero and Deep Mind groups \u2014 discovered an exploitable stack buffer underflow in SQLite, a widely used open source database engine.<\/span><\/p>\n

Specifically, Big Sleep discovered a pattern in the code of a publicly released version of SQLite that creates a potential edge case that needs to be handled by all code that uses the field, the researchers noted. A function in the code failed to correctly handle the edge case, “resulting in a write into a stack buffer with a negative index when handling a query with a constraint on the ‘rowid’ column,” thus creating an exploitable flaw, according to the post.<\/span><\/p>\n

Google reported the bug to SQLite developers in early October. They fixed it on the same day and before it appeared in an official release of the database, so users were not affected.<\/span><\/p>\n

Related:<\/span>News Desk 2024: Hacking Microsoft Copilot Is Scary Easy<\/a><\/p>\n

Inspired by AI Bug-Hunting Peers<\/h2>\n

“We believe this is the first public example of an AI agent finding a previously unknown exploitable memory-safety issue in widely used real-world software,” the Big Sleep team wrote in the post. While this may be true, it’s not the first time an LLM-based reasoning system autonomously found a flaw in the SQLite database engine, Google acknowledged.<\/span><\/p>\n

An LLM model called Atlantis from a group of AI experts called Team Atlanta discovered six zero-day flaws in SQLite3 and even autonomously identified and patched one of them during the AI Cyber Challenge organized by ARPA-H, DARPA, and the White House, <\/span>the team revealed<\/a><\/span> in a blog post in August.<\/span><\/p>\n

In fact, the Big Sleep team used one of the Team Atlanta discoveries \u2014 of “a null-pointer dereference” flaw in SQLite \u2014  to inspire them to use AI “to see if we could find a more serious vulnerability,” according to the post.<\/span><\/p>\n

Software Review Goes Beyond Fuzzing<\/h2>\n

Google and other software development teams already use a process called <\/span>fuzz-testing<\/a><\/span>, colloquially known as “fuzzing,” to help find flaws in applications before release. <\/span>Fuzzing<\/a><\/span> is an approach that targets the software with deliberately malformed data \u2014 or inputs \u2014 to see if it will crash so they can investigate and fix the cause.<\/span><\/p>\n

Related:<\/span>Privacy Anxiety Pushes Microsoft Recall AI Release Again<\/a><\/p>\n

In fact, Google earlier this year released <\/span>an AI-boosted fuzzing framework<\/a><\/span> as an open source resource to help developers and researchers improve how they find software vulnerabilities. The framework automates manual aspects of fuzz-testing and uses LLMs to write project-specific code to boost code coverage.<\/span><\/p>\n

While fuzzing “has helped significantly” to reduce the number of flaws in production software, developers need a more powerful approach “to find the bugs that are difficult (or impossible) to find” in this way, such as variants for previously found and patched vulnerabilities, the Big Sleep team wrote.<\/span><\/p>\n

“As this trend continues, it’s clear that fuzzing is not succeeding at catching such variants, and that for attackers, manual variant analysis is a cost-effective approach,” the team wrote in the post.<\/span><\/p>\n

Moreover, variant analysis is a better fit for current LLMs because its provides them with a starting point \u2014  such as the details of a previously fixed flaw \u2014 for a search, and thus removes a lot of ambiguity from AI-based vulnerability testing, according to Google. In fact, at this point in the evolution of LLMs, lack of this type of starting point for a search can cause confusion, they noted.<\/span><\/p>\n

Related:<\/span>OWASP Releases AI Security Guidance<\/a><\/p>\n

“We’re hopeful that AI can narrow this gap,” the Big Sleep team wrote. “We think that this is a promising path towards finally turning the tables and achieving an asymmetric advantage for defenders.”<\/span><\/p>\n

Glimpse Into the Future<\/h2>\n

Google Big Sleep is still in its research phase, and using AI-based automation to identify software flaws overall is a new discipline. However, there already are tools available that developers can use to get a jump on finding vulnerabilities in software code before public release.<\/span><\/p>\n

Late last month, researchers at Protect AI released Vulnhuntr, a free, open source static code analyzer tool that <\/span>can find zero-day vulnerabilities<\/a><\/span> in Python codebases using Anthropic’s <\/span>Claude artificial intelligence (AI) model<\/a><\/span>.<\/span><\/p>\n

Indeed, Google’s discovery shows promising progress for the future of using AI to help developers troubleshoot software before letting flaws seep into production versions.<\/span><\/p>\n

“Finding vulnerabilities in software before it’s even released means that there’s no scope for attackers to compete: the vulnerabilities are fixed before attackers even have a chance to use them,” Google’s Big Sleep team wrote.<\/span><\/p>\n

Source<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"

Google has discovered its first real-world vulnerability using an artificial<\/p>\n","protected":false},"author":12,"featured_media":6076,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[1],"tags":[809],"class_list":["post-6075","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-uncategorized","tag-dark-reading"],"featured_image_urls":{"full":["https:\/\/i0.wp.com\/ddi.mohflo.net\/wp-content\/uploads\/2024\/11\/google-big-sleep-ai-agent-puts-sqlite-software-bug-to-bed.jpg?fit=1800%2C1013&ssl=1",1800,1013,false],"thumbnail":["https:\/\/i0.wp.com\/ddi.mohflo.net\/wp-content\/uploads\/2024\/11\/google-big-sleep-ai-agent-puts-sqlite-software-bug-to-bed.jpg?resize=150%2C150&ssl=1",150,150,true],"medium":["https:\/\/i0.wp.com\/ddi.mohflo.net\/wp-content\/uploads\/2024\/11\/google-big-sleep-ai-agent-puts-sqlite-software-bug-to-bed.jpg?fit=300%2C169&ssl=1",300,169,true],"medium_large":["https:\/\/i0.wp.com\/ddi.mohflo.net\/wp-content\/uploads\/2024\/11\/google-big-sleep-ai-agent-puts-sqlite-software-bug-to-bed.jpg?fit=640%2C360&ssl=1",640,360,true],"large":["https:\/\/i0.wp.com\/ddi.mohflo.net\/wp-content\/uploads\/2024\/11\/google-big-sleep-ai-agent-puts-sqlite-software-bug-to-bed.jpg?fit=640%2C360&ssl=1",640,360,true],"1536x1536":["https:\/\/i0.wp.com\/ddi.mohflo.net\/wp-content\/uploads\/2024\/11\/google-big-sleep-ai-agent-puts-sqlite-software-bug-to-bed.jpg?fit=1536%2C864&ssl=1",1536,864,true],"2048x2048":["https:\/\/i0.wp.com\/ddi.mohflo.net\/wp-content\/uploads\/2024\/11\/google-big-sleep-ai-agent-puts-sqlite-software-bug-to-bed.jpg?fit=1800%2C1013&ssl=1",1800,1013,true],"chromenews-featured":["https:\/\/i0.wp.com\/ddi.mohflo.net\/wp-content\/uploads\/2024\/11\/google-big-sleep-ai-agent-puts-sqlite-software-bug-to-bed.jpg?fit=1024%2C576&ssl=1",1024,576,true],"chromenews-large":["https:\/\/i0.wp.com\/ddi.mohflo.net\/wp-content\/uploads\/2024\/11\/google-big-sleep-ai-agent-puts-sqlite-software-bug-to-bed.jpg?resize=825%2C575&ssl=1",825,575,true],"chromenews-medium":["https:\/\/i0.wp.com\/ddi.mohflo.net\/wp-content\/uploads\/2024\/11\/google-big-sleep-ai-agent-puts-sqlite-software-bug-to-bed.jpg?resize=590%2C410&ssl=1",590,410,true]},"author_info":{"display_name":"Dark Reading","author_link":"https:\/\/ddi.mohflo.net\/index.php\/author\/darkreading\/"},"category_info":"Uncategorized<\/a>","tag_info":"Uncategorized","comment_count":"0","jetpack_featured_media_url":"https:\/\/i0.wp.com\/ddi.mohflo.net\/wp-content\/uploads\/2024\/11\/google-big-sleep-ai-agent-puts-sqlite-software-bug-to-bed.jpg?fit=1800%2C1013&ssl=1","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/ddi.mohflo.net\/index.php\/wp-json\/wp\/v2\/posts\/6075","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/ddi.mohflo.net\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/ddi.mohflo.net\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/ddi.mohflo.net\/index.php\/wp-json\/wp\/v2\/users\/12"}],"replies":[{"embeddable":true,"href":"https:\/\/ddi.mohflo.net\/index.php\/wp-json\/wp\/v2\/comments?post=6075"}],"version-history":[{"count":0,"href":"https:\/\/ddi.mohflo.net\/index.php\/wp-json\/wp\/v2\/posts\/6075\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/ddi.mohflo.net\/index.php\/wp-json\/wp\/v2\/media\/6076"}],"wp:attachment":[{"href":"https:\/\/ddi.mohflo.net\/index.php\/wp-json\/wp\/v2\/media?parent=6075"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/ddi.mohflo.net\/index.php\/wp-json\/wp\/v2\/categories?post=6075"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/ddi.mohflo.net\/index.php\/wp-json\/wp\/v2\/tags?post=6075"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}