Programming

17666 readers

361 users here now

Welcome to the main community in programming.dev! Feel free to post anything relating to programming here!

Cross posting is strongly encouraged in the instance. If you feel your post or another person's post makes sense in another community cross post into it.

Hope you enjoy the instance!

Rules

Follow the programming.dev instance rules
Keep content related to programming in some way
If you're posting long videos try to add in some form of tldr for those who don't want to watch videos

Wormhole

Follow the wormhole through a path of communities [email protected]

founded 2 years ago

MODERATORS

snowe

Ategon

[email protected]

[Discussion] Devs and Devops: if you have running containers in production, how are you upgrading those containers? (lemdit.com)

submitted 1 year ago by [email protected] to c/programming

14 comments fedilink hide all child comments

Hey all!

I'm seeking guidance. I have a lot of apps that use ORMs like Mongoose or Sequelize or Sqlalchemy, or even just init-db scripts with raw SQL. Point is a lot of apps have changes to the data layer. When we program and make apps they tend to describe how they need to see a database. Sometimes its no trivial feat to do an upgrade - and even if you have a CI/CD pipeline in place what does that look like to have something like even lemmy upgrade in a container.

If you have these apps in production how are you handling these apps? Both developer perspective and devops welcomed.

I see the devs offering insight into maybe how the backend recognizes database state, while the devops perhaps either describing process and implementation.

you are viewing a single comment's thread
view the rest of the comments

[–] JWolf 6 points 1 year ago (1 children)

For me(programmer) it usually depends on the upgrade.

The system I am working on contains like 5-10 containers (depending on customer needs).

In generel I try to have as few connections between services as possible, Low Coupling, High Cohesion. No 2 containers share a database scheme, because that makea database updates way more difficult. The 2 services have to talk to each other through a welldefined API which can be versioned.

Application bugfix Sending out a bugfix usually only effects the actions of a single container. Then we upload the new docker image to the server, shuts down the old instance and start the new container, but reuses all the settings of the old container. This does result in like 10-20 seconds downtime, which is completly acceptable in my case.
Application feature When we implement a big new feature we usually also needs to update the database (new tables/columns). First take a backup of the database before the upgrade. Then like with the bugfix upload and start the new container. In our case we have the application push out the database update (C# dotnet EF core). If it is an even bigger update we may have multiple docker containers that is upgraded at the same time.
Database version upgrade We check the release note for the specific database on how to upgrade it, and follow those steps.
Other utility containers (Nginx, Elastic search, ...) Same as database, check the release note for breaking changes to make sure the upgrade goes well.

Always always always make sure to test the update in a controlled environment that resembles the production environemnt as closely as possible. (maybe even a copy of production if I am lucky)

[–] [email protected] 2 points 1 year ago

This also assumes you have adequate testing capabilities which are important so that you can evaluate unit capabilities, integrated capabilities, and performance characteristics.

On cloud you might have your testing infra be ephemeral and spin up whenever you need to perform the testing (as well as your IAC capabilities).

But this would partially evaluate any issues with the app coordinating with the database.

I would also assume DBA's can write some test scripts to execute as well.