this post was submitted on 01 Aug 2023
525 points (82.2% liked)
Technology
59581 readers
3293 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related content.
- Be excellent to each another!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, to ask if your bot can be added please contact us.
- Check for duplicates before posting, duplicates may be removed
Approved Bots
founded 1 year ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
These biases have always existed in the training data used for ML models (society and all that influencing the data we collect and the inherent biases that are latent within), but it’s definitely interesting that generative models now make these biases much much more visible (figuratively and literally with image models) to the lay person
But they know the AI's have these biases, at least now, shouldn't they be able to code them out or lessen them? Or would that just create more problems?
Sorry, I'm no programer so I have no idea if thats even possible or not. Just sounds possible in my head.
You don't really program them, they learn from the data provided. If say you want a model that generates faces, and you provide it with say, 500 faces, 470 of which are of black women, when you ask it to generate a face, it'll most likely generate a face of a black woman.
The models are essentially maps of probability, you give it a prompt, and ask it what the most likely output is given said prompt.
If she had used a model trained to generate pornography, it would've likely given her something more pornographic, if not outright explicit.
You've also kind of touched on a point of problem with large language models; they're not programmed, but rather prompted.
When it comes to Bing Chat, Chat GPT and others, they have additional AI agents sitting alongside them to help filter/mark out problematic content both provided by the user, as well as possible problematic content the LLM itself generates. Like this prompt, the model marked my content as problematic and the bot gives me a canned response, "Hi, I'm bing. Sorry, can't help you with this. Have a nice day. :)"
These filters are very crude, but are necessary because of problems inherent in the source data the model was trained on. See, if you crawl the internet for data to train it on, you're bound to bump into all sorts of good information; Wikipedia articles, Q&A forums, recipe blogs, personal blogs, fanfiction sites, etc. Enough of this data will give you a well rounded model capable of generating believable content across a wide range of topics. However, you can't feasibly filter the entire internet, among all of this you'll find hate speech, you'll find blogs run by neo nazis and conspiracy theorists, you'll find blogs where people talk about their depression, suicide notes, misogyny, racism, and all sorts of depressing, disgusting, evil, and dark aspects of humanity.
Thus there's no code you can change to fix racism.
But rather simple measures that read the user/agent interaction, filtering it for possible bad words, or likely using another AI model to gauge the probability of an interaction being negative,
As an aside, if she'd prompted "professional Asian woman" it likely would've done a better job. Depending on how much "creative license" she gives the model though, it still won't give her her own face back. I get the idea of what she's trying to do, and there's certainly ways of acheiving it, but she likely wasn't using a product/model weighted to do specifically the thing she was asking to do.
Edit
Just as a test, because I myself got curious; I had Stable Diffusion generate 20 images given the prompt
20 sampling steps, using DPM++ 2M SDE Karras, and the
v1-5-pruned-emaonly
Stable Diffusion model.Here's the result
I changed the prompt to
And here is the result
The models can generate non-white men, but it is in a way just a reflection of our society. White men are the default. Likewise if you prompt it for "loving couple" there'll be so many images of straight couples. But don't just take my word for it, here's an example.
The Hands/digits...the horror....
It's clearly biased against fingered folks!
It can do faces quite well on second passes but struggles hard with hands.
Corporate photography tends to be uncanny and creepy to begin with, so using an AI to generate it made it even more so.
I totally didn’t just spend 30 minutes generating corporate stock photos and laughing at the creepy results. 😅
Just glad there's a tell, for AI photo. Hope it never figure it out.
This is well-stated. As an addendum, if you were to use something like the RevAnimated model (a fairly popular one), it would bias toward digital images rather than photos for essentially the same reason. If you used one of the Anything models, it would generate anime-like images for the same reason. It is pushing toward (one of) the default “understandings” it has of whatever the prompt is. The main issue in the article is the student had both a.) unrealistic expectations about what the AI would do — without being trained on her and her specifically, it wouldn’t know what parameters to hold. It’s not a magic box where you can put in an unknown image and simple manipulate it, at least not in the way one might expect from Photoshop and b) a useless prompt, reflecting a more magical understanding of what the AI would do. It gave her exactly what she asked for, which was (the model’s interpretation) of a person’s linkedin photo using the base image as its target.
That said there’s an exceptionally valid question that’s being sort of asked around and is implied by your post which is what does the model understand the “default” to be. Obviously this aspect isn’t demographically balanced in any way, which is the main problem. That problem, however, isn’t really demonstrated by this example in the way people are thinking. I’d frame the issue as — if she’d asked for a professional LinkedIn photo, she would definitely have gotten a photo of a woman approximately that age and that ethnicity and whatever it thinks a professional linkedin photo is, but it probably still wouldn’t have really looked like her (because it doesn’t know who, in terms of weights, she is); the problem is that without prompting it does default to its default weights of assuming that a generic female is probably white. But a solution would either look like it choosing a totally random nationality every time (which is not what she wanted) or some other specific nationality every time (as some of the more Asian-biased models do) due to her poor prompting.
You put it really succinctly.
These models are trained on images generally depicting a lot of concepts. If I do a simple prompt like
monkey
I'm probably going to get a lot of images of monkeys in a natural setting, because when trained it's given images essentially stating that this is an image of "monkey, tree, fruit, foliage, green" and so forth. Over time this pattern repeats, and it will start associating one concept with another. Because a lot of pictures that depictmonkey
also depictfoliage, nature
and what have you, it makes sense for it to fill in the rest ofmonkey
withfoliage, green, nature
and lessvolcano, playful dog, ice cream
since those concepts haven't been very prominent when presented withmonkey
.That is essentially the problem.
Here is the result for "monkey"
And here is
monkey, volcano, playful dog, ice cream
The datasets are permeated with these kinds of examples. If you prompt for
nurse
you'll get a lot of women in blue/hospitaly clothing, inside hospitals or non-descript backgrounds, and very few men.Here's
photo of a nurse
The more verbose and specific you get though, the likelier it is that you'll get the outcome you want. Here for example is
male (nurse:1) (wearing white scrubs:1) with pink hair skydiving
This was so outlandish that without the
(wearing white scrubs:1)
it just put the skydiver in a random pink outfit, even with the added weight onwearing white scrubs
it has a tendency to put the subject in something pink. Without the extra weight on(nurse:1)
it gave me generic pink (mostly) white men.If we were to fix the biases present in our society, we'd possibly see less biases in the datasets, and subsequently less bias in the final models. The issue I believe, isn't so much a technological one, as it is a societal one. The machines are only racist because we are teaching them to be, by example.
More to the point, there are so many parameters that can be tweaked, to throw your image into a “generator” without knowing what controls you have, what the prompt is doing, what model it’s using etc is like saying “the internet” is toxic because you saw a webpage that had a bad word on it somewhere.
I put her actual photo into SD 1.5 (the same model you used) with 30 step Euler, 5.6 cfg and 0.7 noise and got these back. I’d say 3/4 of them are Asian (and the model had a 70% chance to influence that away if it were truly “biased” in the way the article implies), obviously none of them look like the original lady because that’s not how it works. You could generate a literally infinite number of similar-looking women who won’t look like this lady with this technique.
The issue isn’t so much that the models are biased — that is both tautologically obvious and as mentioned previously, probably preferred (rather than just randomly choosing anything not specified at all times — for instance, your monkey prompt didn’t ask for forest, so should it have generated a pool hall or plateau just to fill something in? The amount of specificity anyone would need would be way too high; people might be generated without two eyes because it wasn’t asked for, for instance); it is that the models don’t reflect the biases of all users. It’s not so much that it made bad choices but that it made choices that the user wouldn’t have made. When the user thinks “person”, she thinks “Asian person” because this user lives in Asia and that’s what her biases toward “person” start with, so seeing a model biased toward people from the Indian subcontinent doesn’t meet her biases. On top of that, there’s a general potential impossibility of having some sort of generic “all people” model given that all people are likely to interpret its biases differently.
With a much lower denoising value I was able to get basically her but airbrushed. It does need a higher denoising value in order to achieve any sort of "creativity" with the image though, so at least with the tools and "skill" I have with said tools, there's a fair bit of manual editing needed in order to get a "professional linkedin" photo.
Yeah if she wants the image to be transformed lower denoising won’t really do it.
Honestly, I know what she did, because I had the same expectation out of the system. She threw in an image and was expecting to receive an infinite number of variations of specifically her but in the style of a “LinkedIn profile photo”, as though by providing the single image, it would map her face to a generic 3d face and then apply that in a variety of different poses, lighting situations and clothing. Rather, what it does is learn the relations between elements in the photo combined with a healthy amount of static noise and then work its way toward something described in the prompt. With enough static noise and enough bias in the model, it might interpret lots of fuzzy stuff around her eyes as “eyes”, but specifically Caucasian eyes since it wasn’t specified in the prompt and it just sees noise around eyes. It’s similarly easy to get a model like Chillout (as someone mentioned in another thread) to bias toward Asian women.
(This is the same picture, just a model change, same parameters. Prompt is: “professional quality linkedin profile photo of a young professional”)
After looking at a number of different photos it’s also easy to start to see where the model is overfit toward a specific look, which is a problem in a technical sense (in that it’s just bad at generating a variety of people, which is probably the intention of the model) and in an ethical sense (in that it’s also subjectively biased).
You could get a more controlled result if you use inpainting. I resized the image in an image editor first, because it gave me really strange results when I gave it the strange resolution of the image from the article. I masked out her face and had the model go ham on the rest of the image, while leaving the face and the hairline untouched.
After that I removed the mask, changed the denoising strength to 0.2, applied the "face fix", and this is the end result.
It's usable, but I think it's a weird use-case for these kinds of tools. Like yeah you can retouch photos if you're careful enough, but at the same time, wouldn't it be easier to just dress up and take some new photos? I dunno, the idea of using an AI generated profile image feels kind of sketchy to me, but then again we have had beautification filters available for ages - my work phone, a Google Pixel 6, comes with it built into the camera application. Every time the camera opens on my face I get this weird uncanny feeling.
Anyway. The article does touch upon a problem that definitely worries me too
I really hope no company would use an image model to analyse candidates profile photos, because as an ugly person that makes me want to scream. However, this has been a problem in the past, Amazon developed a tool for use by recruiters, which turned out to have a bias against women. I can easily see a "CV analysis tool" having a bias against people with names of non-European origin for example.
At this point I think it's impossible to put the genie back into the bottle, given the chance I definitely would, but I think all we can do now is ensure that we try and mitigate potential harm caused by these tools as much as possible.
Indeed, there seems to be some confusion about the wording too. She wrote instructions, like she was instructing a state-of-the-art LLM, "please alter this photo to make it look professional", but the AI can't understand sentence structure or instructions, it just looks for labels that match pictures. So the AI sees "photo, professional" and it sees her starting photo, and it alters the starting photo to produce something that resembles "photo, professional". It doesn't know what those other words mean.
That's not how it works. You don't just "program out the biases" you have to retain the model with more inclusive training data.
even then, it will always move towards the "average" of all combined training data, unless prompted otherwise.
No, that's not how it works either.
If you can code it, it isn't really AI.
AI is able to make the connections when given the data by itself. The problem is that the data required is usually enormous, so the quantity of data is more valued than the quality.
thx I'm gonna use that sentence so much now, I'm so tired of hearing people calling themselves AI experts when they merely do regular programing, like yes your program is pretty cool, but no it's not AI.
"but... but it reacts to its environment, it's intelligent" NO BECAUSE THEN LITERALLY EVERYTHING WOULD BE "INTELLIGENT" and the word AI would be useless, as we have been doing that for decades with if/else
sory im angery
No need to be to be sorry to me.
Shouldn't they be able to code them out?
You can't "code them out" because AI isn't using a simple script like traditional software. They are giant nested statistical models that learn from data. It learns to read the data it was trained on. It learns to understand images that it was trained on, and how they relate to text. You can't tell it "in this situation, don't consider race" because the situation itself is not coded anywhere. It's just learned behaviors from the training data.
Shouldn't they be able to lessen them?
For this one the answer is YES. And they DO lessen them as much as they can. But they're training on data scraped from many sources. You can try to curate the data to remove racism/sexism, but there's no easy way to remove bias from data that is so open ended. There is no way to do this in an automated way besides using an AI model, and for that, you need to already have a model that understands race/gender/etc bias, which doesn't really exist. You can have humans go through the data to try to remove bias, but that introduces a ton of problems as well. Many humans would disagree on what is biased. And human labelers also have a shockingly high error rate. People are flat out bad at repetitive tasks.
And even that only covers data that actively contains bigotry. In most of these generative AI cases, the real issue is just a lack of data or imbalanced data from the internet. For this specific article, the user asked to make a photo look professional. Training data where photos were clearly a professional setting probably came from sites like LinkedIn, which had a disproportionate number of white users. These models also have a better understanding of English than other languages because there is so much more training data available in English. So asian professional sites may exist in the training data, but the model didn't understand the language as well, so it's not as confident about professional images of Asians.
So you can address this by curating the training data. But this is just ONE of THOUSANDS and THOUSANDS of biases, and it's not possible to control all of them in the data. Often if you try to correct one bias, it accidentally causes the model to perform even worse on other biases.
They do their best. But ultimately these are statistical models that reflect the existing data on the internet. As long as the internet contains bias, so will AI
Thank you so much for taking the time to answer!
It’s possible sure. In order to train these image AIs you essentially feed them a massive amount of pictures as “training data.” These biases happen because more often than not the training data used is mostly pictures of white people. This might be due to racial bias of the creators, or a more CRT explanation where they only had the rights to pictures of mostly white people. Either way, the fix is to train the AI on more diverse faces.
There are LoRAs available (hundreds, maybe thousands) to tweak the base model so you can generate exactly what you want. So, problem solved for quite a while now.