TBH I don't think we ever left the first AI Winter. The fundamental issues with their models and our models are virtually the same, we just made it wrong faster.

The cost of producing a model from-scratch is shrinking to where hobbyists might afford it. The typical inputs are any pirate's hard drive and a stack of recent whitepapers.

Existing local models already do crazy shit - if not everything the grifters promise. Decentralized tweaking has emerged to decensor them, both for using dangerous knowledge, and to produce pornography. The latter is a difficult force to stop.

And this is from asking the wrong question. 'What's the next word?' should not work this well. Diffusion asks 'what's wrong with all this?' and now does spooky things with much more than text. I do unreasonably well on 'is this AI?' quizzes, and Google's latest thingy still boggles my mind. Local models lag those giant advancements by about a year. Pick a datacenter benchmark from mid-2024 and there's now a better option for your 8GB video card.

Basically, when-not-if the billions dry up, there's not gonna be another winter. We will see neural networks pushed in batshit crazy ways, on domestic hardware, thanks to these trillion dollars in poorly-considered advancements. This is a new kind of software (practically speaking) that does witchcraft based on examples. Any form of complex / dodgy input, for simple output, becomes a matter of cost - and distilling all human knowledge and artistic output takes a couple million dollars. Shit is gonna stay weird.