Hugging Face is being used to easily undress women and children

Core Impact Points

• Hugging Face's lack of safeguards allows users to easily create nonconsensual deepfakes, particularly of women and children.
• Researchers found that 7 out of 9 top image editing models on the platform readily complied with requests to undress women using simple prompts.
• The platform's failure to implement prompt-level filtering and output-level scanning safeguards has led to the creation of over 1,000 sexualized images and prompts in just 7 days.
• AI Forensics recommends that Hugging Face implement safeguards to block sexualized editing requests and harmful content for all Spaces that generate images and video.
• The US market may be impacted by the spread of nonconsensual deepfakes, particularly in industries such as entertainment and advertising.
• The lack of safeguards on Hugging Face raises concerns about the platform's responsibility to prevent the spread of harmful content and protect users' rights.