this post was submitted on 19 Oct 2024
39 points (84.2% liked)

Programming

17661 readers
216 users here now

Welcome to the main community in programming.dev! Feel free to post anything relating to programming here!

Cross posting is strongly encouraged in the instance. If you feel your post or another person's post makes sense in another community cross post into it.

Hope you enjoy the instance!

Rules

Rules

  • Follow the programming.dev instance rules
  • Keep content related to programming in some way
  • If you're posting long videos try to add in some form of tldr for those who don't want to watch videos

Wormhole

Follow the wormhole through a path of communities [email protected]



founded 2 years ago
MODERATORS
39
You're overcomplicating production (paravoce.bearblog.dev)
submitted 2 months ago* (last edited 2 months ago) by [email protected] to c/programming
 

Sharing some lessons I learned from 10 years/millions of users in production. I’ll be in the comments if anyone has any questions!

top 16 comments
sorted by: hot top controversial new old
[–] [email protected] 37 points 2 months ago (1 children)

Fully agree with the sentiment, but the blog post itself is kinda crap. All it really says is - hey, we're overcomplicating things, but subscribe to my rss for when I actually start talking about it!

[–] [email protected] 7 points 2 months ago (1 children)

Fair criticism. I wanted to lay the groundwork as I intend for it to be a pretty large resource for people over time. Like starting with chapter one before I write the whole book. I hope you can find some value in some of the stuff to come.

[–] [email protected] 4 points 2 months ago

Perhaps some kind of teaser that hints at what your strategy is before going into full detail?

[–] ericjmorey 16 points 2 months ago (1 children)

This is a teaser for the promised future posts. Don't ghost us.

[–] [email protected] 10 points 2 months ago

Writing the second post now :-)

[–] MajorHavoc 16 points 2 months ago* (last edited 2 months ago) (1 children)

You make some great points, but I'm concerned that your preferred solutions may ignore the needs of working with peers. When I've worked with similar solutions before, we had a lot of on call, and it all went to the same person, regardless of who actually answered the phone.

There's nuance to be had in the middle ground:

  • The CI/CD pipeline should deliver releases, but shouldn't be the only way to put them into production. People often get this wrong and have to invoke CI/CD to even do a roll back. Putting something into production without CI/CD should be possible, but it should be loud, to avoid nasty surprises, later.
  • Infrastructure as code is great. It just happens to be backwards, today. We could all go back to point and click changes, if the infrastructure had full journaling (including who made the change) of changes and full rollback capability. Ironically, I expect k8s (or a fork of it) to finally deliver this.
  • the only nice thing I have to say about K8s today is that it's not locking myself into a proprietary cloud, or worse, VMWare. Today, K8s is towering mere centimeters above even worse solutions, all of them objectively awful. I'm not mad at anyone using any of these. We all know we want the freedom of K8s, but no one wants the bullshit interfaces it currently comes with. It will get better.

Anyway, interesting read. Thank you. The only way the current awful state of hosting is going to improve is by having this conversation.

I keep hearing "most people aren't ready for K8S". But there's no such thing. There's just whether K8S (or whatever replaces it) is ready for most people's use cases.

[–] [email protected] 11 points 2 months ago* (last edited 2 months ago) (1 children)

I’m concerned that your preferred solutions may ignore the needs of working with peers. When I’ve worked with similar solutions before, we had a lot of on call, and it all went to the same person, regardless of who actually answered the phone.

Totally hear you and have the same experience myself. The approach I'm advocating for is simply running a binary on a server with rsync to deploy, and architecting your product around that limitation. Teaching a team the basics of Linux sysadmin will be incredibly useful for their careers, and it's something that the whole team can easily learn. Then you don't need to hire a k8s team -- any engineer can do some basic debugging when things go sideways.

[–] MajorHavoc 7 points 2 months ago

The approach I'm advocating for is simply running a binary on a server with rsync to deploy, and architecting your product around that limitation.

Intriguing!

I'm looking forward to your blog series on this.

[–] [email protected] 12 points 2 months ago* (last edited 2 months ago)

I've been doing this for 30+ years and it seems like the push lately has been towards oversimplification on the user side, but at the cost of resources and hidden complexity on the backend.

As an Assembly Language programmer I'm used to programming with consideration towards resource consumption. Did using that extra register just cause a couple of extra PUSH and POP commands in the loop? What's the overhead on that?

But now some people just throw in a JavaScript framework for a single feature and don't even worry about how it works or the overhead as long as the frontend looks right.

The same is true with computing. We're abstracting containers inside of VMs on top of base operating systems which is adding so much more resource utilization to the mix (what's the carbon footprint on that?) with an extremely complex but hidden backend. Everything's great until you have to figure out why you're suddenly losing packets that pass through a virtualized router to linuxbridge or OVS to a Kubernetes pod inside a virtual machine. And if one of those processes fails along the way, BOOM! it's all gone. But that's OK; we'll just tear it down and rebuild it.

I get it. I understand the draw, and I see the benefits. IaC is awesome, and the speed with which things can be done is amazing. My concern is that I've seen a lot of people using these things who don't know what's going on under the hood, so they often make assumptions or mistakes that lead to surprises later.

I'm not sure what the answer is other than to understand what you're doing at every step of the way, and always try to choose the simplest route (but future-proofed).

[–] Kissaki 10 points 2 months ago

learned from 10 years/millions of users in production

10 years per millions of users is an interesting metric :P

[–] ericjmorey 6 points 2 months ago (1 children)

How are you liking bearblog.dev?

[–] [email protected] 7 points 2 months ago

Only just started using it, but I love it. Simple, basic blogging without the enshittification of Medium.

[–] [email protected] 2 points 2 months ago

Looking forward to reading more about it.

[–] nullPointer 2 points 2 months ago (1 children)

I am a big advocate of the KISS strategy. look forward to seeing more of what you got.

[–] [email protected] 1 points 2 months ago* (last edited 2 months ago) (1 children)

In addition to this I like YAGNI: you ain’t gonna need it

Don’t implement features you don’t need because you think it might/could/should be useful in the future. YAGNI

[–] robinm 1 points 2 months ago

DRY and YAGNI are awesome iif you also practice YNIRN (You Need It Right Now)! Otherwise you just get boilerplate of spaghetti