this post was submitted on 18 Sep 2024
350 points (93.1% liked)

Technology

58133 readers
4730 users here now

This is a most excellent place for technology news and articles.


Our Rules


  1. Follow the lemmy.world rules.
  2. Only tech related content.
  3. Be excellent to each another!
  4. Mod approved content bots can post up to 10 articles per day.
  5. Threads asking for personal tech support may be deleted.
  6. Politics threads may be removed.
  7. No memes allowed as posts, OK to post as comments.
  8. Only approved bots from the list below, to ask if your bot can be added please contact us.
  9. Check for duplicates before posting, duplicates may be removed

Approved Bots


founded 1 year ago
MODERATORS
top 50 comments
sorted by: hot top controversial new old
[–] [email protected] 5 points 1 hour ago

have we tried feeding them actual human beings yet ?

[–] [email protected] 1 points 1 hour ago

Two outcasts among their peers, Gary Wallace and Wyatt Donnelly spent a good deal of their youth as pioneers and early adopters of AI.

[–] [email protected] 32 points 8 hours ago (3 children)

Every single one of us, as kids, learned the concept of "garbage in, garbage out"; most likely in terms of diet and food intake.

And yet every AI cultist makes the shocked pikachu face when they figure out that trying to improve your LLM by feeding it on data generated by literally the inferior LLM you're trying to improve, is an exercise in diminishing returns and generational degradation in quality.

Why has the world gotten both "more intelligent" and yet fundamentally more stupid at the same time? Serious question.

[–] Croquette 1 points 26 minutes ago

Because the dumdums have access to the whole world at the tip of the fingertip without having to put any efforts in.

In a time without that, they would be ridiculed for their stupid ideas and told to pipe down.

Now they can find like minded people and amplify their stupidity, and be loud about it.

So every dumdum becomes an AI prompt engineer (whatever the fuck that means) and know how to game the LLM, but do not understand how it works. So they are basically just snake oil salesmen that want to get on the gravy train.

[–] [email protected] 3 points 2 hours ago* (last edited 2 hours ago)

Remember Trump every time he's weighed in on something, like suggesting injecting people with bleach, or putting powerful UV lights inside people, or fighting Covid with a "solid flu vaccine" or preventing wildfires by sweeping the forests, or suggesting using nuclear weapons to disrupt hurricane formation, or asking about sharks and electric boat batteries? Remember these? These are the types of people who are in charge of businesses, they only care about money, they are not particularly smart, they have massive gaps in knowledge and experience but believe that they are profoundly brilliant and insightful because they've gotten lucky and either are good at a few things or just had an insane amount of help from generational wealth. They have never had anyone, or very few people genuinely able to tell them no and if people don't take what they say seriously they get fired and replaced with people who will.

[–] [email protected] 24 points 8 hours ago

Because the people with power funding this shit have pretty much zero overlap with the people making this tech. The investors saw a talking robot that aced school exams, could make images and videos and just assumed it meant we have artificial humans in the near future and like always, ruined another field by flooding it with money and corruption. These people only know the word "opportunity", but don't have the resources or willpower to research that "opportunity".

[–] [email protected] 4 points 8 hours ago

Oh no. Anyways...

[–] [email protected] 20 points 13 hours ago* (last edited 13 hours ago)

oh no are we gonna have to appreciate the art of human beings? ew. what if they want compensation‽

[–] [email protected] 8 points 12 hours ago* (last edited 12 hours ago) (1 children)

If mainstream blogs are writing about it, what would make someone think that AI companies haven't thoroughly dissected the problem and are already working on filtering out AI fingerprints from the training data set? If they can make a sophisticated LLM, chances are they can find methods to XOR out generated content.

[–] [email protected] 7 points 8 hours ago (1 children)

What would make me think that they haven't "thoroughly dissected" it yet is that I'm a skeptic, and since I'm a skeptic I don't immediately and without evidence believe that every industry is capable of identifying, dissecting, and solving every problem with its products.

[–] [email protected] 2 points 3 hours ago

Ironically given their skillset, training an ML model on known and properly tagged AI generated and non-AI-generated stuff might actually work.

[–] [email protected] 7 points 13 hours ago (1 children)
[–] [email protected] 7 points 12 hours ago
[–] [email protected] 13 points 18 hours ago (1 children)

Fake news, just like that one time Nightshade "killed" stable diffusion (literally had no effect) Flux came out not long ago and it's better than ever

[–] [email protected] 1 points 5 hours ago (1 children)

At this point the synthetic data is good enough to intentionally be used for training LLMs.

[–] [email protected] 3 points 5 hours ago

Yeah, just filter out the bad generated images and feed the good ones again, until the model learns how to produce only good ones.

[–] [email protected] 9 points 17 hours ago (2 children)

is it not relatively trivial to pre-vet content before they train it? at least with aigen text it should be.

[–] RvTV95XBeo 21 points 17 hours ago (2 children)

The problem is these AI companies currently exist on the business model of not paying for information, and that generally includes not wanting to pay content curators.

Google is probably the only one in a position to potentially outsource by making everyone solve a "does this hand look normal to you" CAPTCHA

They can try and train AI to detect AI, but that's also difficult.

load more comments (2 replies)
[–] [email protected] 1 points 12 hours ago

It depends on what you are looking for. Identifying AI generated data is generally hard, though it can be done in specific cases. There is no mathematical difference between the 1s and 0s that encoded AI generated data and any other data. Which is why these model collapse ideas are just fantasy. There is nothing magical about any data that makes it "poisonous" to AI. The kernel of truth behind these ideas is not likely to matter in practice.

[–] [email protected] 3 points 13 hours ago

Sooner or later it is supposed to happen, but I don't think we are quite there....Yet.

[–] [email protected] 9 points 18 hours ago (7 children)

Kind of like how true thoughts and opinions on complex topics are boiled down to digestible concepts for others to understand who then perpetuate those concepts without understanding them and the meaning degrades and we dont think anymore, just repeat stuff in social media comments.

Side note... this article sucks and seems like it was ai generated. Repetitive and no author credit? Just says it was originally posted elsewhere.

Generative AI isnt in danger of being killed as this clickbait titled suggests... just hindered.

load more comments (7 replies)
[–] pastermil 12 points 20 hours ago

More like degenerative AIs

[–] [email protected] 102 points 1 day ago (1 children)

It is their own fault for poisoning the internet with their slop.

[–] [email protected] 51 points 1 day ago (6 children)

In case anyone doesn't get what's happening, imagine feeding an animal nothing but its own shit.

[–] [email protected] 8 points 19 hours ago

Photocopy of a photocopy is my go-to metaphor for model collapse.

load more comments (5 replies)
[–] [email protected] 162 points 1 day ago (11 children)

Let's go, already!

How you can help: If you run a website and can filter traffic by user agent, get a list of the known AI scrapers agent strings and selectively redirect their requests to pre-generated AI slop. Regular visitors will see the content and the LLM scraper bots will scrape their own slop and, hopefully, train on it.

load more comments (11 replies)
[–] [email protected] 27 points 1 day ago (1 children)

Old news? Seems to be a subject of several papers for some time now. Synthetic data has been used successfully already for very specific domains.

[–] [email protected] 3 points 17 hours ago

Yup, old news and wrong news. Also so many people who hate AI but don't understand how it works. Pretty disappointing for a technology community.

[–] [email protected] 36 points 1 day ago (1 children)

…………………. Good?

[–] [email protected] 1 points 13 hours ago

Tbh I'm a bit lost on the purpose of this

[–] [email protected] 5 points 18 hours ago (1 children)

"Model collapse" is just a fancy way of saying "our stupid ideas are bad and nobody wants them."

load more comments (1 replies)
[–] [email protected] 64 points 1 day ago (2 children)

Model collapse is just a euphemism for “we ran out of stuff to steal”

[–] [email protected] 30 points 1 day ago* (last edited 1 day ago)

It's more ''we are so focused on stealing and eating content, we're accidently eating the content we or other AI made, which is basically like incest for AI, and they're all inbred to the point they don't even know people have more than two thumb shaped fingers anymore."

load more comments (1 replies)
[–] [email protected] 75 points 1 day ago (2 children)

More like... Degenerative AI *ba dum tsss

load more comments (2 replies)
[–] [email protected] 17 points 1 day ago

The solution for this is usually counter training. Granted my experience is on the opposite end training ai vision systems to id real objects.

So you train up your detector ai on hand tagged images. When it gets good you use it to train a generator ai until the generator is good at fooling the detector.

Then you train the detector on new tagged real data and the new ai generated data. Once it's good at detection again you train the generator ai on the new detector.

Repeate several times and you usually get a solid detector and a good generator as a side effect.

The thing is you need new real human tagged data for each new generation. None of the companies want to generate new human tagged data sets as it's expensive.

load more comments
view more: next ›