this post was submitted on 22 Nov 2024
752 points (98.1% liked)

Comic Strips

12943 readers
2872 users here now

Comic Strips is a community for those who love comic stories.

The rules are simple:

Web of links

founded 2 years ago
MODERATORS
752
submitted 3 weeks ago* (last edited 3 weeks ago) by [email protected] to c/[email protected]
you are viewing a single comment's thread
view the rest of the comments
[–] [email protected] 1 points 3 weeks ago* (last edited 3 weeks ago)

It turns out these clusters are being used very inefficiently, seeing how Qwen 2.5 was trained with a fraction of the GPUs and is clobbering models from much larger clusters.

One could say Facebook, OpenAI, X and such are "hoarding" H100s but are not pressured to utilize them efficiently since they are so GPU unconstrained.

Google is an interesting case, as Gemini is getting better quickly, but they presumably use much more efficient/cheap TPUs to train.