このエピソードについて
In May of last year, a warning about AI came from somewhere unexpected: The Auschwitz-Birkenau State Museum.
Posting publicly on social media, the museum warned about a Facebook account using generative AI to create fake images of people who died in the Holocaust. The people in said images were sometimes real—with real names, birthplaces, and stories of deportation that the Auschwitz-Birkenau State Museum itself had shared before. They had real faces captured in real surviving photographs, which were likely abused to generate the false images.
In other words, someone, or some team of people online, was deepfaking the Holocaust.
As the Auschwitz museum wrote online:
“These are not real photos of the victims. They are digital inventions, often stylized or sanitized, that risk turning remembrance into fictionalized performance. The history of Auschwitz is a well-documented story. Altering its visual record with AI imagery introduces distortion, no matter the intent.”
Months later, the public found out what that intent was: money.
A BBC investigation found an international network of Facebook accounts posting AI-generated images to earn money from those images’ potential virality. It’s a problem sometimes referred to as “AI slop” but it comes with a major incentive. When accounts that make these kinds of images are invited to Facebook’s content monetization program, they can make $1,000 a month for posting anything that gets clicks.
And on Facebook, the BBC found, that means several accounts posting AI-generated images about the Holocaust. As the BBC reported:
“AI spammers have posted fake images purporting to be from inside [Auschwitz], such as a prisoner playing a violin or lovers meeting at the boundaries of fences—attracting tens of thousands of likes and shares.”
The economics of lying are concrete today. People can use AI to make fake images that make people feel good about terrible things or feel scared about untrue things, and they can make money until shut down by the Big Tech platforms themselves, which, in this case, only happened because of the BBC’s investigation. In fact, it’s that type of inaction from social media platforms that compelled the German government and multiple Holocaust memorial institutions to send an open letter earlier this year that asked for better controls and restrictions against this type of content.
As the signatories warned in their letter, the economic appeal for these accounts to distort history is too high a risk to allow. You can read the full letter here.
Today, on the Lock and Code podcast with host David Ruiz, we speak with Clara Mansfeld, a historian working on digital communications at one of the institutions signed onto the open letter—the Foundation of Hamburg Memorials and Learning Centers Commemorating the Victims of Nazi Crimes. In their conversation, Mansfeld discusses digital access to history, the manipulation of factual records through AI-generated imagery, and the threat that society faces when it becomes harder to evaluate the truth.
“What happens when the first thought we have with every historical image is, ‘Is that even real or is that AI?’ I don’t think we have really grasped what that means for us as a society.”
Tune in today.
You can also find us on Apple Podcasts, Spotify, and whatever preferred podcast platform you use.
For all our cybersecurity coverage, visit Malwarebytes Labs at malwarebytes.com/blog.
Show notes and credits:
Intro Music: “Spellbound” by Kevin MacLeod (incompetech.com)
Licensed under Creative Commons: By Attribution 4.0 License
http://creativecommons.org/licenses/by/4.0/
Outro Music: “Good God” by Wowa (unminus.com)
Listen up—Malwarebytes doesn't just talk cybersecurity, we provide it.
Protect yourself from online attacks that threaten your identity, your files, your system, and your financial well-being with our exclusive offer for Malwarebytes Premium for Lock and Code listeners.
