Google is set to release new guidelines for AI app developers on Google Play to address concerns about inappropriate or banned content. Apps offering AI features will be required to prevent generating restricted content like sexual content and violence while also giving users a way to flag offensive material. Developers must thoroughly test their AI tools to ensure user security and privacy are respected.
Google will crack down on apps with marketing materials endorsing inappropriate use cases. Apps that promote stripping people naked or creating non-consensual nude images, even if the capabilities are not real, risk being banned from Google Play. These measures come in response to a rise in AI stripping apps, with reports of AI-faked nudes causing problems in schools and online harassment.
The company hopes these policies will keep harmful AI-generated content away from Google Play. It advises developers to consult the AI-Generated Content Policy for app approval requirements. Apps must not allow the creation of restricted content and should enable users to report inappropriate material, especially in apps where user interactions shape content and experience.
Developers are cautioned against violating Google Play’s rules regarding app promotion. Inappropriate use cases could result in an app being removed from the store. Protecting apps against manipulation that could lead to harmful content is also emphasized, with developers urged to use Google’s closed test feature to gather feedback from users before launch.
Google is also offering resources like its People + AI guide to assist developers in creating AI applications responsibly. These measures aim to address the misuse of AI technology for creating harmful or offensive content, particularly in light of recent incidents involving AI deepfakes. By following these guidelines, developers can help ensure a safer and more secure environment for users on Google Play.
Article Source
https://techcrunch.com/2024/06/06/google-play-cracks-down-on-ai-apps-after-circulation-of-apps-for-making-deepfake-nudes/