The Dark Side of AI Image Generators

I want to be upfront about something before I write this: I am not a person who thinks AI image generators are purely bad or that everyone who uses them is doing something wrong. I use Midjourney. I've used Stable Diffusion. I've paid for Firefly credits. I'm not writing this from the outside looking in.

But I've been using these tools long enough now that I've had to actually sit with some stuff that made me genuinely uncomfortable. And I think if you use these tools, you should know what I know.

The training data problem is not hypothetical

Here's the thing that took me a while to fully understand: these models didn't learn to make art from nothing. They learned by looking at hundreds of millions of images scraped from the internet. Paintings, photographs, illustrations, concept art, basically anything publicly posted online got swept up and used to train the model, whether the person who made it agreed to that or not.

I have a friend who does concept art professionally. She's been doing it for about eight years, has a really distinct style, posts her work online. She didn't consent to her work being in any training set. She didn't get paid. Nobody asked her. And you can prompt certain models in ways that clearly reflect her aesthetic influences without her ever knowing.

Now, some people argue this is like how human artists learn, you look at work, you absorb it, you develop your own style. And I think that argument has some validity! But there's also a difference between a person looking at paintings to learn and a corporation scraping millions of images without permission to build a commercial product. The scale is different. The commercial intent is different. The lack of any compensation mechanism is different.

The specific artists being harmed

This isn't just abstract. There's a list, literally called Have I Been Trained, where you can enter your name and see if your work was in LAION, one of the big datasets used to train a lot of these models. Many artists have found their work there without any memory of consenting to it.

Some artists have seen their names used as style prompts so frequently that it's directly affected their ability to get commissions. Why hire the artist when you can prompt for "in the style of [artist name]" and get something close enough for the client? I don't think everyone doing that is evil, but I do think it's worth sitting with the impact.

Greg Rutkowski is probably the most famous example. He's a Polish fantasy artist with a gorgeous, very distinctive painterly style. For a while his name was one of the most commonly used style prompts in Stable Diffusion, more than Picasso, more than Van Gogh. He started getting fewer commissions. People were generating work that looked like his without him involved at all. He has spoken publicly about how much this has affected his livelihood.

That's not a theoretical harm. That's a real person whose work got absorbed into a model and then competed against him.

The deepfake problem isn't separate from this

Okay so this one makes me genuinely angry and I'm not going to pretend otherwise.

The same technology that makes it easy to generate a pretty landscape or a cool character portrait also makes it incredibly easy to put real people's faces into images they never agreed to. I'm not going to describe the worst versions of this because you already know what they are. But the fact that the exact same tools are used for both the cute and the horrifying is something I think about a lot.

Several of the popular consumer image generators have had to build filters specifically to stop people from generating non-consensual imagery of real people. Some of those filters work pretty well. Some don't work well at all. And the open-source versions of these models have basically no safeguards by design.

The people most affected by this are women, overwhelmingly. Celebrities, but also regular people whose photos exist somewhere online. It's a harm that falls unevenly and the people building these tools have been frustratingly slow to take it seriously.

What the companies are actually doing about it

This varies a lot and I think it matters who you give your money to.

Adobe Firefly was specifically trained on licensed images and public domain content. They made a business decision to do it the expensive and complicated way so that their model could be used commercially without the same consent problems. I respect that, and it's part of why I use it for client work specifically.

Midjourney has been less transparent about their training data. They've also fought against artist opt-out requests and were sued by a group of artists in 2023. I still use Midjourney because I think the output quality for certain things is genuinely in a different category. But I'm not under any illusion that they've handled this well.

Some of the newer models have announced opt-in training approaches or compensation programs for artists. Whether those actually work at scale or are mostly PR, I genuinely don't know yet. I'm watching to see how they develop.

I don't think the answer is "stop using it"

I've thought about this a lot and I don't think refusing to use any AI image tools is the right call, at least not for me. These tools exist. They're being used. My not using them doesn't undo the training data situation or protect the artists whose work is already in the models.

But I do think it matters where I spend money. I try to use Firefly for things that need to be commercially clean. I pay attention to which companies are actually working on consent and compensation mechanisms versus which ones are just hoping nobody notices. I don't prompt for specific living artists' styles by name, that one feels like a clear line to me personally.

And I talk about this stuff. Because I think a lot of people who use these tools haven't thought through where they came from, and that's not entirely their fault, the companies selling these products have not been eager to explain it.

The tools are genuinely incredible and also genuinely complicated. Both of those things are true at the same time.