I want to buy a new GPU mainly for SD. The machine-learning space is moving quickly so I want to avoid buying a brand new card and then a fresh model or tool comes out and puts my card back behind the times. On the other hand, I also want to avoid needlessly spending extra thousands of dollars pretending I can get a 'future-proof' card.
I'm currently interested in SD and training LoRas (etc.). From what I've heard, the general advice is just to go for maximum VRAM.
- Is there any extra advice I should know about?
- Is NVIDIA vs. AMD a critical decision for SD performance?
I'm a hobbyist, so a couple of seconds difference in generation or a few extra hours for training isn't going to ruin my day.
Some example prices in my region, to give a sense of scale:
- 16GB AMD: $350
- 16GB NV: $450
- 24GB AMD: $900
- 24GB NV: $2000
edit: prices are for new, haven't explored pros and cons of used GPUs
Good to know about CUDA/Direct ML.
I found a couple of 2022 posts recommending 3090s, especially since cryptocoin miners were selling lots of them cheap. Thanks for the heads up about the 5000 release, I suspect it will be above my budget but it will net me better deals on a 4090 :P
DirectML sucks but ROCm is great, but you need to check if the software you want to use works with ROCM. Also note there's only like 4 cards that work with ROCm as well.
Yeah I don't think 4090 is going down in price. As of now, they're more expensive than when they launched and it seems production is ramping down.