I did Gentoo Stage 1 (which was very similiar to what you plan to do) in 2005 with a shitty laptop. 24 hours until I had a working shell compiled. A whole week until I had a graphical desktop working properly. Stupid me didn't have enough and did it again in 2013 with better hardware within just 36 hours to the desktop.
If you seek a challenge that leaves you with angelic patience once you've overcome the never ending rages you'll encounter to push through to the end against all odds, lots of errors, bad documentation, dependencies from hell AND keeping it running, which will inevitably raise your patience muscles strength again and again, then yes, do it. Just accept that at some point, something will break inside you.