Microsoft and others prohibit the use of their generative AI systems to create a variety of content. Off-limits content includes content that features or promotes sexual exploitation or abuse, erotic or pornographic content, or content that is based on race, ethnicity, national origin, gender, gender identity, sexual orientation, religion, Contains content that attacks, defames, or excludes people based on age. Disability condition or similar characteristics. Additionally, you may not create content that threatens, intimidates, encourages physical harm, or otherwise contains abusive behavior.
In addition to explicitly prohibiting the use of such platforms, Microsoft will inspect both the prompts entered by users and their resulting output to ensure that the requested content violates any of these terms. We have also developed guardrails to check for signs of this. These codebase restrictions have been repeatedly bypassed in recent years, both by benign hacks by researchers and by malicious actors.
Microsoft did not explain exactly how the defendant’s software was allegedly designed to circumvent the guardrails it created.
Masada writes:
Microsoft’s AI services introduce strong safeguards, including built-in safety mitigations at the AI model, platform, and application level. As alleged in a court filing released today, Microsoft has developed sophisticated software that allows a group of foreign-based attackers to exploit exposed customer credentials harvested from public websites. I observed what you were doing. In doing so, they attempted to identify and gain unauthorized access to accounts using certain generative AI services and intentionally modify the functionality of those services. Cybercriminals then used these services and resold access to other malicious actors along with detailed instructions on how to use these custom tools to generate harmful and illegal content. After the discovery, Microsoft revoked the cybercriminal’s access, took action, and strengthened safeguards to further block such malicious activity in the future.
The complaint alleges that the defendants’ services violate the Computer Fraud and Abuse Act, the Digital Millennium Copyright Act, the Lanham Act, the Racketeer Influenced and Corrupt Organizations Act, and include wire fraud, access device fraud, common law violations, and torts. They claim that it constitutes interference. The complaint seeks an injunction restraining the defendants from engaging in “any activities hereunder.”
