this post was submitted on 23 Nov 2024

354 points (89.2% liked)

Technology

59669 readers

2830 users here now

This is a most excellent place for technology news and articles.

Our Rules

Follow the lemmy.world rules.
Only tech related content.
Be excellent to each another!
Mod approved content bots can post up to 10 articles per day.
Threads asking for personal tech support may be deleted.
Politics threads may be removed.
No memes allowed as posts, OK to post as comments.
Only approved bots from the list below, to ask if your bot can be added please contact us.
Check for duplicates before posting, duplicates may be removed

Approved Bots

founded 1 year ago

MODERATORS

[email protected]

354

OpenAI, Google, Anthropic admit they can’t scale up their chatbots any further (pivot-to-ai.com)

submitted 4 days ago* (last edited 4 days ago) by [email protected] to c/[email protected]

86 comments fedilink hide all child comments

I'm usually the one saying "AI is already as good as it's gonna get, for a long while."

This article, in contrast, is quotes from folks making the next AI generation - saying the same.

(page 2) 38 comments

sorted by: hot top controversial new old

[–] [email protected] 9 points 4 days ago* (last edited 4 days ago) (1 children)

Though, I don't think that means they won't get any better. It just means they don't scale by feeding in more training data. But that's why OpenAI changed their approach and added some reasoning abilities. And we're developing/researching things like multimodality etc... There's still quite some room for improvements.

[–] [email protected] 3 points 3 days ago

Though, I don't think that means they won't get any better. It just means they don't scale by feeding in more training data.

Agreed. There's plenty of improvement to be had, but the gravy train of "more CPU or more data == better results" sounds like it's ending.

[–] [email protected] 8 points 4 days ago* (last edited 4 days ago)

It's a known problem - though of course, because these companies are trying to push AI into everything and oversell it to build hype and please investors, they usually try to avoid recognizing its limitations.

Frankly I think that now they should focus on making these models smaller and more efficient instead of just throwing more compute at the wall, and actually train them to completion so they'll generalize properly and be more useful.

[–] [email protected] 8 points 4 days ago (2 children)

I smell a sentient AI trying to throw us off it's plans for world domination..

[–] [email protected] 4 points 3 days ago* (last edited 3 days ago)

Everyone ignore this comment please. I'm quite human. I have the normal 7 fingers (edit: on each of my three hands!) and everything.

[–] [email protected] 3 points 4 days ago (1 children)

Cylons. I knew it.

[–] [email protected] 0 points 2 days ago

Can't be, I haven't fucked one yet, and everyone knows Cylonism is an STD.

Unless I'm an Eskimo brother and don't know it...

[–] [email protected] 5 points 4 days ago (5 children)

Looks, like AI buble is slowly coming to end just like what happned to crypto and NFT buble.

load more comments (5 replies)

[–] [email protected] -1 points 3 days ago (1 children)

They might be right but I read some of the linked articles on this blog (?), the authors just come off as not really knowing much about current AI technologies, and at the same time very very arrogant.

[–] [email protected] -1 points 2 days ago

The article talks about LLM developers / operators. Not sure how you got from that to "current AI technologies" - a completely unrelated topic.

[+] [email protected] -6 points 4 days ago (1 children)

I believe that the current LLM paradigm is a technological dead end. We might see a few additional applications popping up, in the near future; but they'll be only a tiny fraction of what was promised.

My bet is that they'll get superseded by models with hard-coded logic. Just enough to be able to correctly output "if X and Y are true/false, then Z is false", without fine-tuning or other band-aid solutions.

[–] [email protected] 9 points 4 days ago (2 children)

Seems unlikely as that's essentially what we had before and they were not very good at all.

[–] [email protected] 4 points 3 days ago

If you're referring to symbolic AI, I don't think that the AI scene will turn 180° and ditch NN-based approaches. Instead what I predict is that we'll see hybrids - where a symbolic model works as the "core" of the AI, handling the logic, and a neural network handles the input/output.

[–] [email protected] 3 points 3 days ago* (last edited 3 days ago)

Unlikely, but there's some percedent.

We've seen this pattern play out in video games a bunch of times.

Revolutionary new way to do things. It's cool, but not... You know...fun.

So we give up on it as a dead and and go back to the old ways for awhile.

Then somebody figures out how to (usually hard code) bumpers on the new revolutionary new way, such that it stays fun.

Now the revolutionary new way is the new gold stand and default approach.

For other industries, replace "fun" above with the correct goal for than industry. "Profitable" is one that the AI hucksters are being careful not to say...but "honest", "correct" and "safe" also come to mind.

We are right before the bit where we all decide it was a bad idea.

Which comes before we figure out hard-coding the bumpers can get us where we wanted, after a lot of work by really smart well paid humans.

I've seen industries skip the "all decide it was a bad idea" phase, and go straight to the "hard work by humans to make this fulfill the available promise" phase, but we don't actually look on track to, today.

Many current investors are convicned that their clever talking puppet is going to do the hard work of engineering the next generation of talking puppet.

I have some faith that we can reach that milestone. I'm familiar enough with the current generation of talking puppet to confidently declare that this won't be the time it happens.

My incentive in sharing all this is that I like over half of you reading there, and so figure I can give some of you a shot at not falling for this particular "investment phase" which is essentially, in practical terms, a con.

load more comments