this post was submitted on 19 Oct 2023
539 points (96.5% liked)
Technology
60047 readers
3338 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related content.
- Be excellent to each another!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, to ask if your bot can be added please contact us.
- Check for duplicates before posting, duplicates may be removed
Approved Bots
founded 2 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
Sam Altman (Creator of the freakish retina scanning based Worldcoin) would agree, it seems. The current path for LLMs and GPT seems to be in something of a bind, because to seriously improve upon what it currently does it needs to do something different, not more of the same. And figuring out something different could be very hard. https://www.wired.com/story/openai-ceo-sam-altman-the-age-of-giant-ai-models-is-already-over/
At least that's what I understand of it.
He's not saying "AI is done, there's nothing else to do, we've hit the limit", he's saying "bigger models don't necessarily yield better results like we had initially anticipated"
Sam recently went before congress and advocated for limiting model sizes as a means of regulation, because, at the time, he believed bigger would generally always mean better outputs. What we're seeing now is that if a model is too large it will have trouble producing truthful output, which is super important to us humans.
And honestly, I don't think anyone should be shocked by this. Our own human brains have different sections that control different aspects of our lives. Why would an AI brain be different?
Future of AI is definitely going towards Manager/Agent model. It allows for an AI to handle all the tasks without keeping it to one model or method. We’re already seeing this with ChatGPT using Mathematica for math questions. Soon we can see art AI using different models and methods based on text input.
I gather that this is partly because data sizes haven't been going up with model sizes. That is likely to change soon as synthetic data starts to overtake organic data in both quantity and quality.