this post was submitted on 28 Feb 2024
2 points (62.5% liked)
Framework Laptop Community
2574 readers
6 users here now
Related links:
- Framework website: https://frame.work/
- Official Framework Mastodon: https://fosstodon.org/@frameworkcomputer
Related communities:
- Buy it for life community: [email protected]
- Fairphone community: [email protected]
founded 3 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
I would like to retract the first sentence. It is misleading - the model can run without any hiccups with ROCm, and is probably a decent GPU for this job. However, I will provide more context on the second sentence.
I tried training and running models (GAN, LLM) on my Lenovo IdeaPad S540 (i5-8265U + 8GB + MX250 2GB laptop). The result of such heavy computation is that the battery is toast, the barrel port adapter is severely damaged where it connects with the board, the metallic panels have stress marks due to uneven heating, and the soldered RAM is a goner. And the battery has no capacity. Remember those old TVs where you had to hit them to make it work? That's the state of my device right now.
The cooling system in a laptop isn't the best. So, it would be better to run the model on a desktop or cloud GPU. By the way, older stable diffusion models seem to work on something as old as RX580 with ROCm.
I see. Thank you.
Would keeping it plugged in or removing the battery help with the battery issue?
Edit: Also, is there any way to force the GPU to throttle earlier to reduce damage?
I think that would still be a bad idea - running billions of parameters on a laptop - because it's not just the battery, you might also want to think about how much the heat-sink can handle cooling. However, there's nothing wrong with running the model a few times. If that were a desktop GPU, it would handle a load as heavy as that easily.
Might I suggest you try using KOALA once? It has way lesser parameters than your typical large-scale diffusion model, so it will be forgiving on your device. Best of luck with your attempt.