This week, Google launched a new reward program named Google’s AI Bounty Program, designed to find bugs in AI products. This program identifies what constitutes an AI bug, breaking it down as problems that leverage a large language model or a generative AI system to cause harm or exploit a security loophole. Based on the severity of the issue, Google offers up to $30k. It encompassess altering someone’s account or data to invade their privacy or do something unwanted, like one flaw exposed in the past. Let’s dig deeper into this matter and discuss what Google’s AI Bounty program brings.
What is an AI Bug for Google?
Google has hinted at the type of vulnerabilities that could be considered for the researchers. Let’s suppose an attacker fools Google Home into unlocking a smart door or using a secret command that makes Gmail summarize someone’s emails and send them to a third party. These types of high-risk flaws are ones that Google wants to disclose before they can be utilized in the real world.
Nevertheless, the company has clarified one thing that if Gemini or any other AI model simply hallucinates, it does not consider as a bug. Issues related to the type of content the AI generates, such as hate speech or copyrighted resources, must be reported directly in the product using the feedback tools. Google opines that this helps its AI safety teams to refine and improve models in a more targeted manner. Previously, we talked about Twitter’s Bug Bounty Programs that can help you better understand.
Security Flaws Listed by Google
- Rogue actions that Google defines as ‘attacks which alter the state of the victim’s account or data with a proper security implication. For example, there could be an indirect prompt injection attack that takes place when a user injects malicious instructions into a prompt upon which the model can act. This leads Google Home to do something like ‘unlock a smart lock’.
- Sensitive data exfiltration, which discloses an individual’s PII or other sensitive information without user approval. This can also include an indirect prompt injection attack, where an AI system summarizes the contents of others’ emails and then shares it with an attacker-controlled account.
- Continuous cross-user HTML injection on a Google app that does not have a user-generated content warning and presents a promising phishing attack vector at the panel’s discretion. In simple terms, it is a security issue if someone uses a product to share an attacker-generated website that fools a Google tool without a user-generated content warning and then shares it for phishing attacks.
- Model theft is another concern that involves attackers exfiltrating entire and confidential model parameters.
- Context manipulation is also counted as a security flaws that allow for continuous, ongoing, and secret manipulation of the context of a victim’s AI development. A specific example of this is when an attacker sends a scheduled invite to a victim that causes storage in an AI product.
- Access control bypass flaws are where the attackers bypass access controls and steal data that could not be accessible other way.
- Unauthorized product access or allowing Google server-side features on the user’s account is also a flaw without purchasing them.
- Cross-user denial of service that includes a continuous rejection of service for an AI product or particular feature in a victim account.
- Apart from this, the attacks are in scope. The products are categories into three tiers, including flagship, standard, and other. The flagship products encompass Google Search, Gemini apps, and Google Workspace core apps.
- Standard apps include high-sensitivity tools like AI Studio, Jules, and Google Workspace apps such as NotebookLM, AppSheet.
What is the Reward?
The biggest reward that you can earn is up to $20k for the vulnerabilities identified in Google’s key products, such as Search, Gmail, Gemini apps, and Drive. If a report scores most for its quality or originality, bonuses can go up to $30k. However, you can expect lower yet meaningful rewards in Google’s AI Bounty Program if the flaws are traced in tools like NotebookLM or the experimental AI assistant Jules.
As per Google, the Google’s AI Bounty Program is associated with its wider effort to support AI security since these systems become deeply embedded into the products. In fact, the tech giant says researchers have already earned over $430,000 in the last two years by discovering AI-related vulnerabilities, even before the launch of this official program. Though Google is recently involved in an AI Overview lawsuit that must not be forgotten, and this could be a way to divert people.
Launch of CodeMender Tool
Apart from Google’s AI Bounty Program, the company has also launched a tool named CodeMender. It is an AI agent developed to spot and resolve security risks in open-source software automatically. To date, CodeMender has helped detect over 70 verified issues after human review. The company suggests that tools like CodeMental demonstrate how AI can play a significant role in making technology safer. Stay tuned for more updates on this tool.
Also Read:
