A laptop is a great place to start.

I like using desktop components as I've been able to incrementally upgrade the ram, CPU, and drives as the years go by. A lot of people also really like using single board computers.

The only thing I'd recommend against are pre-built NASes. Theyre proprietary AF and so overpriced for what you get if you don't need the handholding of the consumer NAS software.

One thing I recommend doing, is keeping step by step notes on everything you set up, and keep a list of files and folders you'd need to keep to easily run whatever you're running on a new system.

That way, moving to a new system, changing your config, or reinstalling the OS is so much easier. A couple years down the line you'll be thanking yourself for writing down how the hell you configured that one thing years back.

Almost every problem I've had was due to me not accounting for some quirk of my config that I'd forgotten about.

And that would apply with a VPS, too, if you end up going that route.