test the easy things - having a test suite that passes feels good, having some tests makes it easier to add more later, and testing utility code is usually pretty easy and important
make the important decisions correctly, the rest can be sloppy - e.g. decide early if you need async or multi-threading and get the signatures of your core code right, then take shortcuts on the rest
don't refactor unless it's blocking new features - it's easy to get sucked into "perfect code," so leave notes instead of doing the refactor

Basically, follow the 80/20 rule, get 80% of the important stuff right for 20% of the effort. Once everything is working, add more tests and refactor while getting feedback from users (if applicable).