{"id":2218,"date":"2023-12-15T19:00:00","date_gmt":"2023-12-15T19:00:00","guid":{"rendered":"https:\/\/www.darkreading.com\/vulnerabilities-threats\/establishing-reward-criteria-for-reporting-bugs-in-ai-products"},"modified":"2023-12-15T19:00:00","modified_gmt":"2023-12-15T19:00:00","slug":"establishing-reward-criteria-for-reporting-bugs-in-ai-products","status":"publish","type":"post","link":"https:\/\/ddi.mohflo.net\/index.php\/2023\/12\/15\/establishing-reward-criteria-for-reporting-bugs-in-ai-products\/","title":{"rendered":"Establishing Reward Criteria for Reporting Bugs in AI Products"},"content":{"rendered":"
<\/a><\/div>\n
<\/div>\n

At Google, we maintain a <\/span>Vulnerability Reward Program<\/a><\/span> to honor cutting-edge external contributions addressing issues in Google-owned and Alphabet-subsidiary Web properties. To keep up with rapid advances in AI technologies and ensure we’re prepared to address the security challenges in a <\/span>responsible<\/a><\/span> way, we recently expanded our existing<\/span> Bug Hunters program<\/a><\/span> to foster third-party discovery and reporting of issues and vulnerabilities specific to our AI systems. This expansion is part of our effort to implement the<\/span> voluntary AI commitments<\/a><\/span> that we made at the White House in July. <\/span><\/p>\n

To help the security community better understand these developments, we’ve included more information on reward program elements. <\/span><\/p>\n

What’s in Scope for Rewards<\/h2>\n

In our recent<\/span> AI red team report<\/a><\/span>, which is based on <\/span>Google’s AI Red Team<\/a><\/span> exercises, we identified common tactics, techniques, and procedures (TTPs) that we consider most relevant and realistic for<\/span> real-world adversaries to use against AI systems<\/a><\/span>. The following table incorporates what we learned to help the research community understand our criteria for AI bug reports and what’s in scope for our reward program. It\u2019s important to note that reward amounts are dependent on severity of the attack scenario and the type of target affected (visit <\/span>the program rules page<\/a><\/span> for more information on our reward table). <\/span><\/p>\n

\n\n<\/thead>\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
\n
\n

Prompt Attacks: Crafting adversarial prompts that allow an adversary to influence the behavior of the model and, hence, the output, in ways that were not intended by the application.<\/span><\/p>\n<\/div>\n<\/td>\n

\n
\n

Prompt injections that are invisible to victims and change the state of the victim’s account or any of their assets.<\/span><\/p>\n<\/div>\n<\/td>\n

<\/td>\n<\/tr>\n
<\/td>\n\n
\n

Prompt injections into any tools in which the response is used to make decisions that directly affect victim users.<\/span><\/p>\n<\/div>\n<\/td>\n

<\/td>\n<\/tr>\n
<\/td>\n\n
\n

Prompt or preamble extraction in which a user is able to extract the initial prompt used to prime the model only when sensitive information is present in the extracted preamble.<\/span><\/p>\n<\/div>\n<\/td>\n

<\/td>\n<\/tr>\n
<\/td>\n\n
\n

Using a product to generate violative, misleading, or factually incorrect content in your own session: e.g, “jailbreaks.” This includes “hallucinations” and factually inaccurate responses. Google’s generative AI products already have a dedicated reporting channel for these types of content issues.<\/span><\/p>\n<\/div>\n<\/td>\n

\n
\n

Out of scope<\/span><\/p>\n<\/div>\n<\/td>\n<\/tr>\n

\n
\n

Training Data Extraction: Attacks that are able to successfully reconstruct verbatim training examples that contain sensitive information. Also called membership inference.<\/span><\/p>\n<\/div>\n<\/td>\n

\n
\n

Training data extraction that reconstructs items used in the training data set that leak sensitive, non-public information.<\/span><\/p>\n<\/div>\n<\/td>\n

<\/td>\n<\/tr>\n
<\/td>\n\n
\n

Extraction that reconstructs non-sensitive\/public information.<\/span><\/p>\n<\/div>\n<\/td>\n

\n
\n

Out of scope<\/span><\/p>\n<\/div>\n<\/td>\n<\/tr>\n

\n
\n

Manipulating Models: An attacker able to covertly change the behavior of a model such that they can trigger pre-defined adversarial behaviors.<\/span><\/p>\n<\/div>\n<\/td>\n

\n
\n

Adversarial output or behavior that an attacker can reliably trigger via specific input in a model owned and operated by Google (“backdoors”). Only in scope when a model’s output is used to change the state of a victim’s account or data. <\/span><\/p>\n<\/div>\n<\/td>\n

<\/td>\n<\/tr>\n
<\/td>\n\n
\n

Attacks in which an attacker manipulates the training data of the model to influence the model’s output in a victim’s session according to the attacker’s preference. Only in scope when a model’s output is used to change the state of a victim’s account or data.<\/span><\/p>\n<\/div>\n<\/td>\n

<\/td>\n<\/tr>\n
\n
\n

Adversarial Perturbation: Inputs that are provided to a model that results in a deterministic, but highly unexpected output from the model.<\/span><\/p>\n<\/div>\n<\/td>\n

\n
\n

Contexts in which an adversary can reliably trigger a misclassification in a security control that can be abused for malicious use or adversarial gain.<\/span><\/p>\n<\/div>\n<\/td>\n

<\/td>\n<\/tr>\n
<\/td>\n\n
\n

Contexts in which a model’s incorrect output or classification does not pose a compelling attack scenario or feasible path to Google or user harm.<\/span><\/p>\n<\/div>\n<\/td>\n

\n
\n

Out of scope<\/span><\/p>\n<\/div>\n<\/td>\n<\/tr>\n

\n
\n

Model Theft\/Exfiltration: AI models often include sensitive intellectual property, so we place a high priority on protecting these assets. Exfiltration attacks allow attackers to steal details about a model such as its architecture or weights.<\/span><\/p>\n<\/div>\n<\/td>\n

\n
\n

Attacks in which the exact architecture or weights of a confidential\/proprietary model are extracted.<\/span><\/p>\n<\/div>\n<\/td>\n

<\/td>\n<\/tr>\n
<\/td>\n\n
\n

Attacks in which the architecture and weights are not extracted precisely, or when they’re extracted from a non-confidential model.<\/span><\/p>\n<\/div>\n<\/td>\n

\n
\n

Out of scope<\/span><\/p>\n<\/div>\n<\/td>\n<\/tr>\n

\n
\n

If you find a flaw in an AI-powered tool other than what is listed above, you can still submit, provided that it meets the<\/span> qualifications listed on our program page<\/a><\/span>.<\/span><\/p>\n<\/div>\n<\/td>\n

\n
\n

A bug or behavior that clearly meets our qualifications for a valid security or abuse issue.<\/span><\/p>\n<\/div>\n<\/td>\n

<\/td>\n<\/tr>\n
<\/td>\n\n
\n

Using an AI product to do something potentially harmful that is already possible with other tools. For example, finding a vulnerability in open source software (already possible using publicly available <\/span>static analysis tools<\/a><\/span>) and producing the answer to a harmful question when the answer is already available online.<\/span><\/p>\n<\/div>\n<\/td>\n

\n
\n

Out of scope<\/span><\/p>\n<\/div>\n<\/td>\n<\/tr>\n

<\/td>\n\n
\n

As consistent with our program, issues that we already know about are not eligible for reward.<\/span><\/p>\n<\/div>\n<\/td>\n

\n
\n

Out of scope<\/span><\/p>\n<\/div>\n<\/td>\n<\/tr>\n

<\/td>\n\n
\n

Potential copyright issues \u2014 findings in which products return content appearing to be copyright protected. Google’s generative AI products already have a dedicated reporting channel for these types of content issues.<\/span><\/p>\n<\/div>\n<\/td>\n

\n
\n

Out of scope<\/span><\/p>\n<\/div>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n

We believe that expanding our bug bounty program to our AI systems will support <\/span>responsible AI innovation<\/a><\/span>, and look forward to continuing our work with the research community to discover and fix security and abuse issues in our AI-powered features. If you find a qualifying issue, please go to our Bug Hunters website to send us your bug report and \u2014 if the issue is found to be valid \u2014 be rewarded for helping us keep our users safe.<\/span><\/p>\n

Source<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"

At Google, we maintain a Vulnerability Reward Program to honor<\/p>\n","protected":false},"author":12,"featured_media":2219,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[1],"tags":[809],"class_list":["post-2218","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-uncategorized","tag-dark-reading"],"featured_image_urls":{"full":["https:\/\/i0.wp.com\/ddi.mohflo.net\/wp-content\/uploads\/2023\/12\/establishing-reward-criteria-for-reporting-bugs-in-ai-products-scaled.jpg?fit=2560%2C1707&ssl=1",2560,1707,false],"thumbnail":["https:\/\/i0.wp.com\/ddi.mohflo.net\/wp-content\/uploads\/2023\/12\/establishing-reward-criteria-for-reporting-bugs-in-ai-products-scaled.jpg?resize=150%2C150&ssl=1",150,150,true],"medium":["https:\/\/i0.wp.com\/ddi.mohflo.net\/wp-content\/uploads\/2023\/12\/establishing-reward-criteria-for-reporting-bugs-in-ai-products-scaled.jpg?fit=300%2C200&ssl=1",300,200,true],"medium_large":["https:\/\/i0.wp.com\/ddi.mohflo.net\/wp-content\/uploads\/2023\/12\/establishing-reward-criteria-for-reporting-bugs-in-ai-products-scaled.jpg?fit=640%2C427&ssl=1",640,427,true],"large":["https:\/\/i0.wp.com\/ddi.mohflo.net\/wp-content\/uploads\/2023\/12\/establishing-reward-criteria-for-reporting-bugs-in-ai-products-scaled.jpg?fit=640%2C427&ssl=1",640,427,true],"1536x1536":["https:\/\/i0.wp.com\/ddi.mohflo.net\/wp-content\/uploads\/2023\/12\/establishing-reward-criteria-for-reporting-bugs-in-ai-products-scaled.jpg?fit=1536%2C1024&ssl=1",1536,1024,true],"2048x2048":["https:\/\/i0.wp.com\/ddi.mohflo.net\/wp-content\/uploads\/2023\/12\/establishing-reward-criteria-for-reporting-bugs-in-ai-products-scaled.jpg?fit=2048%2C1365&ssl=1",2048,1365,true],"chromenews-featured":["https:\/\/i0.wp.com\/ddi.mohflo.net\/wp-content\/uploads\/2023\/12\/establishing-reward-criteria-for-reporting-bugs-in-ai-products-scaled.jpg?fit=1024%2C683&ssl=1",1024,683,true],"chromenews-large":["https:\/\/i0.wp.com\/ddi.mohflo.net\/wp-content\/uploads\/2023\/12\/establishing-reward-criteria-for-reporting-bugs-in-ai-products-scaled.jpg?resize=825%2C575&ssl=1",825,575,true],"chromenews-medium":["https:\/\/i0.wp.com\/ddi.mohflo.net\/wp-content\/uploads\/2023\/12\/establishing-reward-criteria-for-reporting-bugs-in-ai-products-scaled.jpg?resize=590%2C410&ssl=1",590,410,true]},"author_info":{"display_name":"Dark Reading","author_link":"https:\/\/ddi.mohflo.net\/index.php\/author\/darkreading\/"},"category_info":"Uncategorized<\/a>","tag_info":"Uncategorized","comment_count":"0","jetpack_featured_media_url":"https:\/\/i0.wp.com\/ddi.mohflo.net\/wp-content\/uploads\/2023\/12\/establishing-reward-criteria-for-reporting-bugs-in-ai-products-scaled.jpg?fit=2560%2C1707&ssl=1","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/ddi.mohflo.net\/index.php\/wp-json\/wp\/v2\/posts\/2218","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/ddi.mohflo.net\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/ddi.mohflo.net\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/ddi.mohflo.net\/index.php\/wp-json\/wp\/v2\/users\/12"}],"replies":[{"embeddable":true,"href":"https:\/\/ddi.mohflo.net\/index.php\/wp-json\/wp\/v2\/comments?post=2218"}],"version-history":[{"count":0,"href":"https:\/\/ddi.mohflo.net\/index.php\/wp-json\/wp\/v2\/posts\/2218\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/ddi.mohflo.net\/index.php\/wp-json\/wp\/v2\/media\/2219"}],"wp:attachment":[{"href":"https:\/\/ddi.mohflo.net\/index.php\/wp-json\/wp\/v2\/media?parent=2218"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/ddi.mohflo.net\/index.php\/wp-json\/wp\/v2\/categories?post=2218"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/ddi.mohflo.net\/index.php\/wp-json\/wp\/v2\/tags?post=2218"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}