this post was submitted on 22 Nov 2024
749 points (98.1% liked)

Comic Strips

12758 readers
3969 users here now

Comic Strips is a community for those who love comic stories.

The rules are simple:

Web of links

founded 1 year ago
MODERATORS
749
submitted 5 days ago* (last edited 5 days ago) by Joker to c/[email protected]
you are viewing a single comment's thread
view the rest of the comments
[–] [email protected] 1 points 2 days ago* (last edited 2 days ago)

It turns out these clusters are being used very inefficiently, seeing how Qwen 2.5 was trained with a fraction of the GPUs and is clobbering models from much larger clusters.

One could say Facebook, OpenAI, X and such are "hoarding" H100s but are not pressured to utilize them efficiently since they are so GPU unconstrained.

Google is an interesting case, as Gemini is getting better quickly, but they presumably use much more efficient/cheap TPUs to train.