0 ℃ isn't "fairly cold outside". It's literally the definition of freezing cold.

...for water. At 1 atmosphere of pressure. Not taking into account salinity.

0 ℉ is "dead" if you're not wearing quite heavy clothing.

Lots of temperatures on both ends of the spectrum are "dead" without proper attire, regardless of what unit of measurement is used.

0 ℃ is "really cold outside" and still understating things.

I think there's a bit of a reference frame issue here.

Zero C is normal winter temperature for a lot of people. For some, it's downright balmy. If it's sunny, I won't need more than a fleece and jeans. Working outside, I'll probably ditch the sleeves after a while.

Going off of your instance, I'm guessing you're in Australia. Since I don't know where, I grabbed a large southern city (Melbourne), and looked up the record holder for coldest temperature (Charlotte Pass). All temps in Celsius:

Melbourne:

Charlotte Pass:

For comparison, here's a city near me (New York), and a random town I picked from a map in northern Minnesota (known for being cold in the winter).

New York:

Roseau:

That... is a stark difference. Where you live makes a huge impact on what feels "normal." 0 C is no big deal here. It's just "cold." Minnesota in January? "What a nice day!" That's not nonsense, it's different perspectives.

Is Fahrenheit arbitrary and outdated? Yes. Is Celsius arbitrary? Also yes. There's nothing special about the freezing and boiling points of water at 1atm. But that's the (basis behind the) current scientific standard. Is it ridiculous that the US still uses Fahrenheit? Yes. Why? I don't run the place, I just live here.

Does any of this matter in the day to day of normal people? No. Will people keep arguing about it? Absolutely.