The primary application at my job was...not well written. Originally .NET Framework 3.5 with a strange collection of approaches to MVC. SQL Server for data. I claimed it look like something written by CS students fresh out of college on their first job. Turned out I was close...it was written by 3rd-year CS interns. We've fixed and improved a ton (including migrating to .NET 6 earlier this year).
One of the ongoing issues was the use of SMALLDATETIME and DATE fields for everything. We've been gradually migrating these to DATETIME2 or DATETIMEOFFSET (we don't have future dates) as UTC. Because 95% of our usage had been CST/EST, we've typically set the time component to 17 hours to get a reasonable noonish time in those zones when we don't have a better time of day.
Had another round of these today and thought it was going well. Finally got it running and everything was +1 day from where it should be. Spent an hour trying to figure out the issue.
Converting for display incorrectly? No. Serializer not doing what was expected? No. Configured time zone got messed up? No. Brought in our DBA to help me figure it out.
I had added 17 days twice.
Side note: I wish everybody was on UTC time and time zones would forever disappear into the ether.
TLDR; Migrated time wrong because I'm stupid.
Timezone programming sucks. The philosophy I got by is, to store/track/log everything in UTC. When it is displayed, then change the format to the local user timezone IF NEEDED. For example if it's like network device logs or security events, one timezone (UTC normally) should be used, not changed for every user. Convert the reverse for input from client, to back end.
This only works out of you're not tracking future dates, but yeah, that's basically what we're doing. Server side is UTC, UI converts into user's zone for display and sends UTC back to the server.
Yeah the future dates (more future times, I guess) is something a lot of people neglect. "Do everything in UTC" is sadly not a magic bullet for all times.
First of all you need to decide if you really want UTC or if you actually want TAI. But if you do want UTC (and you do, a lot of the time), future times become a problem. If they're TAI, then you can't (always) predict exactly when they'll happen. If they're UTC, then you can't (always) predict exactly how far in the future they are.