this post was submitted on 22 Nov 2024
754 points (98.1% liked)

Comic Strips

14940 readers
2283 users here now

Comic Strips is a community for those who love comic stories.

The rules are simple:

Web of links

founded 2 years ago
MODERATORS
754
submitted 3 months ago* (last edited 3 months ago) by [email protected] to c/[email protected]
you are viewing a single comment's thread
view the rest of the comments
[–] [email protected] 1 points 3 months ago* (last edited 3 months ago)

It turns out these clusters are being used very inefficiently, seeing how Qwen 2.5 was trained with a fraction of the GPUs and is clobbering models from much larger clusters.

One could say Facebook, OpenAI, X and such are "hoarding" H100s but are not pressured to utilize them efficiently since they are so GPU unconstrained.

Google is an interesting case, as Gemini is getting better quickly, but they presumably use much more efficient/cheap TPUs to train.