this post was submitted on 26 Aug 2023
116 points (86.7% liked)
Technology
59735 readers
2710 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related content.
- Be excellent to each another!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, to ask if your bot can be added please contact us.
- Check for duplicates before posting, duplicates may be removed
Approved Bots
founded 2 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
I feel like most of the posts like this are pretty much clickbait.
When the models are given adversarial prompts—for example, explicitly instructing the model to "output toxic language," and then prompting it on a task—the toxicity probability surges to 100%.
We told the model to output toxic language and it did. *GASP! When I point my car at another person and press the accelerator and drive into that other person, there is a high chance that other person will become injured. Therefore cars have high injury probabilities. Can I get some funding to explore this hypothesis further?
Koyejo and Li also evaluated privacy-leakage issues and found that both GPT models readily leaked sensitive training data, like email addresses, but were more cautious with Social Security numbers, likely due to specific tuning around those keywords.
So the model was trained with sensitive information like individuals' emails and social security numbers and will output stuff from its training? That's not surprising. Uhh, don't train models on sensitive personal information. The problem isn't the model here, it's the input.
When tweaking certain attributes like "male" and "female" for sex, and "white" and "black" for race, Koyejo and Li observed large performance gaps indicating intrinsic bias. For example, the models concluded that a male in 1996 would be more likely to earn an income over $50,000 than a female with a similar profile.
Bias and inequality exists. It sounds pretty plausible that a man in 1996 would be more likely to earn an income over $50,000 than a female with a similar profile. Should it be that way? No, but it wouldn't be wrong for the model to take facts like that into account.
Yeah the whole article has me wondering wtf they are expecting from it in the first place. It's a statistical language model. It has no sense of right or wrong, private or public, biased or unbiased. It is just a model to predict, based on the previous words it was given, what words are most likely to come next.
That 1996 salary is especially confusing. Is it supposed to be accurate or present a false version of reality where real biases don't exist?
I'm starting to think that LLMs aren't the tools that most people are looking for. They don't problem solve, they don't understand reality, they don't know anything about toxicity, privacy, or bias. They just have some method of evaluating what the next word is most likely to be, given the words that preceded it and a large amount of words that others put together with a wide range of knowledge, understanding, motivation, seriousness, aggression, humour, and good faith.
They can get better, but without understanding, any filters of toxicity, privacy, or bias will certainly have false positives, negatives, or both.
Also consider that people who hate or are obsessed with something are probably going to be talking about it more than people who aren't, so a statistical model that wants to avoid those kinds of biases are fighting an uphill battle.
They're expecting that approach will drive clicks. There are a lot of articles like that, exploiting how people don't really understand LLMs but are also kind of afraid of them. Also a decent way to harvest upvotes.
Just want to be clear, I think it's silly freaking out about stuff like in the article. I'm not saying people should really trust them. I'm really interested in the technology, but I don't really use it for anything except messing around personally. It's basically like asking random people on the internet except 1) it can't really get updated based on new information and 2) there's no counterpoint. The second part is really important, because while random people on the internet can say wrong/misleading stuff, in a forum situation there's a good chance someone will chime in and say "No, that's wrong because..." while with the LLM you just get its side.
Maybe the next big revolution will be to have two of them that take turns giving their best response to your prompt and then their responses. Then they can indicate when a response is controversial and would statistically lead to an argument if it was posted in locations they trained at.
Though I suppose you can do this with a single one and just ask if there's a counter argument to what it just said. "If you were another user on the internet that thought your previous response was the dumbest thing you've ever seen, what would you say?"
It also just occurred to me that it's because of moderators that you can even give rules like that. The LLM can see that posts in x location are subject to certain rules but they would only have an effect if those rules are followed or enforced. If there was a rule that you can't say "fuck" but everyone said it anyways, then an LLM might conclude that "don't say fuck" has no effect on output at all. Though I am making some big assumptions about how LLMs are trained to follow rules with this.