Hugging Face Hosts Thousands of Deepfake Nude Models, Unmoderated

Thousands of models designed to generate nude images of real people were uploaded to hugging face, a platform for distributing machine learning models. The uploads used standard procedures. Nobody flagged them during upload because the flagging system does not know what a deepfake nude is when it sees one, only that a file has been submitted.

This is how moderation at scale fails: the tool does not see the problem because the tool was not built to see it. The acceptable use policy forbids the activity. The activity occurred anyway. The discovery was accidental. This is the normal state now.

Hugging face will issue a statement. Some models will be removed. Others will migrate to other platforms or to private servers. The models themselves are simple enough that they can be rebuilt in hours. The problem is not solved because the problem was not understood as needing solving until after it became visible.