this post was submitted on 24 Sep 2024
71 points (82.6% liked)

Selfhosted

40313 readers
127 users here now

A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.

Rules:

  1. Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.

  2. No spam posting.

  3. Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it's not obvious why your post topic revolves around selfhosting, please include details to make it clear.

  4. Don't duplicate the full text of your blog or github here. Just post the link for folks to click.

  5. Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).

  6. No trolling.

Resources:

Any issues on the community? Report it using the report flag.

Questions? DM the mods!

founded 1 year ago
MODERATORS
 

And it failed spectacularly.

We only needed a simple form, but we wanted to be fancy, so we used "nextcloud forms".

The docker image automatically updated the install to nextcloud 30, but the forms app requires nextcloud 29 or lower. No warning whatsoever. It's an official app, couldn't they wait that it was ready for NC 30 before launching it? The newsletter boasts "NC hub 9 is the best thing after sliced bread" yet i don't see any difference both in visual or performance compared to NC hub 2

Conclusion: we made our business to rely on nextcloud forms as a signup form, but the only reason we were using it was disabled who knows how many weeks ago.

top 50 comments
sorted by: hot top controversial new old
[–] [email protected] 84 points 1 month ago (5 children)

Wait, you update productions systems without running a staging environment? Or even checking the update notes and your installed apps? Also no backups? What kind of business are you running over there?

[–] [email protected] 54 points 1 month ago (8 children)

Oh, Nextcloud docker is a joke. They follow no standards or best practices when it comes to docker. They keep the entire app directory mounted as a volume, which means it does upgrade you without you "needing" to upgrade the docker image. They have volumes within volumes they need to mount. Their configs can (and do) override environment variables. Most actions that need to be taken require running an occ command which can only be done by exec'ing into the container.

Nextcloud docker is honestly just such a joke. They should have rethought their application from a docker sense and they didn't. God just number one - Docker images should never update. It's a freaking pinned version for a reason. If I want to update, it should be as simple as upping the version tag, and it does any upgrades in place when I do that.

I honestly steer people away from Nextcloud now because of how mismanaged their images are.

[–] [email protected] 13 points 1 month ago* (last edited 1 month ago) (1 children)

Yep, and I'd guess there's probably a huge component of "it must be as easy as possible" because the primary target is selfhosters that don't really even want to learn how to set up Docker containers properly.

The AIO Docker image is an abomination. The other ones are slightly more sane but they still fundamentally mix code and data in the same folder so it's not trivial to just replace the app.

In Docker, the auto updater should be completely neutered, it's the wrong way to update the app.

The packages in the Arch repo are legit saner than the Docker version.

[–] [email protected] 2 points 1 month ago (2 children)

I had to learn how to mount subpaths for their terrible container, and god just the updater is mind boggling. And I have to store their code in a volume, because of course I have to, why would code and configuration ever need to be... configurable? I actually just tried to put their config.php into a ConfigMap just to try, and of course PHP doesn't allow that - not that I blame PHP for it - but ffs it's been years, it's time to allow config to also come from a yaml or something.

[–] [email protected] 5 points 1 month ago (5 children)
load more comments (5 replies)
[–] [email protected] 3 points 1 month ago

Having the web server be able to overwrite its own app code is such a good feature for security. Very safe. Only need a path traversal exploit to backdoor config.php!

[–] [email protected] 2 points 1 month ago (1 children)

I wiped a whole drive (luckily it was filled with a redundant backup) with the docker image, as the behavior was (or still is, don't know if it was fixed) to rm -rf . and replace with fresh stuff if occ isn't found. So in the docker compose I accidentally mistyped the wrong volume as /mnt/disk2 instead of /mnt/disk3 and it erased it

[–] [email protected] 4 points 1 month ago

Oh yeah, if you're in a professional environment, I'm sorry but that's just not great. The only way I'd consider running Nextcloud professionally would be on a VM of it's own with nightly disk backups, with blob storage as the backing - and even then with the cloud costs really how close are you to just paying for an enterprise license to Google or Microsoft? Plus the headache of not having to worry about it yourself

load more comments (6 replies)
[–] [email protected] 4 points 1 month ago

One that lacks a good IT department apparently

[–] [email protected] 4 points 1 month ago (2 children)

To be fair a certain security company was in global news for exactly that same send it behavior. Why waste precious resources on multiple instances? Investors hate waste. 😅

load more comments (2 replies)
[–] [email protected] 3 points 1 month ago (1 children)

Yes no staging because it's something used at most by 2 concurrent users, we were ok with 95% reliability (we discovered it was disabled after at least two weeks lol)

Otherwise we would just have signed up to one of the many cloud forms sites at $100/year

Backups daily but it's unthinkable to revert something like nextcloud to a months old one

Subscribed to both newsletter and RSS feed to know about issues (the command to update the docker images isn't automated but manually issued). The maintainer of the forms app is nextcloud itself so any incompatibility should have been written in red bold characters in the blog posts and newsletter.

[–] [email protected] 3 points 1 month ago (1 children)

Why are your backups so out of date? Just setup daily snapshots and call it a day if it isn't critical. You never want to update major versions first thing. Wait 3 months and then update.

This smells like shadow IT

load more comments (1 replies)
[–] [email protected] 3 points 1 month ago

If I understand correctly, nextcloud automatically updated ... which I didn't think it would, normally. Maybe it's a "feature" of the AIO docker image?

[–] [email protected] 32 points 1 month ago (2 children)

Backups and rollbacks should be your next endeavor.

[–] [email protected] 8 points 1 month ago

Seems easier to blame Nextcloud

[–] [email protected] 3 points 1 month ago

I have daily Borg backups held for at least one year but the problem is that the issue came out at least two weeks ago and nobody noticed. It's better to have nothing (customer gets error page when viewing useless survey that nobody is watching) rather to restore such a old backup (everyone loses 2-4 weeks of data)

[–] [email protected] 31 points 1 month ago (8 children)

The docker image automatically updated the install to nextcloud 30, but the forms app requires nextcloud 29 or lower.

Lol. Do not blame others for your incompetence. If you have automatically updates enabled then that is your fault when it breaks things. Just pin the major version with a tag like nextcloud:29 or something. Upgrading major versions automatically in production is a terrible decision.

[–] [email protected] 17 points 1 month ago (5 children)

Docker images should never self update - that's an anti pattern. They should be static code. The only time I would expect a docker image to "auto update" is if I was using the "latest" or "stable" tag and Compose/Kubernetes/I repull the image - but the image should never update itself.

Yes, OP bit off more than they could chew. Nextcloud, however, is breaking the entire purpose of Docker images by having an auto-updater at all.

[–] [email protected] 17 points 1 month ago (5 children)

If you say

Thing:latest

and then redeploy your compose file or what not,

well, you're getting the latest!

load more comments (5 replies)
load more comments (4 replies)
load more comments (7 replies)
[–] [email protected] 20 points 1 month ago (1 children)

Specify a Version Tag in docker compose and update nextcloud deliberately through the webapp, that way it doesn't update automatically on a pull

[–] [email protected] 2 points 1 month ago (4 children)

You can't update it in the web app. You need to do a docker-compose pull followed by docker-compose up -d

load more comments (4 replies)
[–] [email protected] 15 points 1 month ago (1 children)

Docker automatically upgrades if you tell it to by specifying "latest" or not specifying a version number. But it only upgrades if you issue the pull command or the compose up command. There are ways to start without a pull like using start or restart. So yes, there was warning and something you did actively told it to upgrade.

And it's really bad practice to update any software without testing, especially between breaking/major version numbers.

Finally, it's not uncommon for a platform to release its update and then the plugins or addons to follow. Especially with major updates that require lots of testing before release. This allows plugin/add-on makers to fully test their software with the release version of the platform rather than all of the plugin makers having to wait for one that may be lagging behind.

load more comments (1 replies)
[–] [email protected] 14 points 1 month ago (11 children)

Why did you do automatic updates without testing? That is the real issue.

Honestly your IT department sounds like it could use some help

load more comments (11 replies)
[–] [email protected] 9 points 1 month ago (1 children)

There was a recent related discussion on Hacker News and the top comment discusses why this sort of solution is not likely to be the best fit for smaller organizations. In short, doing it well requires time and effort from someone technically sophisticated, who must do more than the bare minimum for good results, as you just learned.

Even then, it's likely to be less reliable than solutions hosted by big corporations and when there's a problem, it's your problem. I don't want to discourage you, but understand what you're committing to and make sure you have adequate buy-in in your organization.

[–] [email protected] 3 points 1 month ago

That reminds me of work. I'm old, young me has been through the mistakes and the pain of wanting to control and self-host everything.

Now I manage a team of young idealists who have not yet been burned sufficiently hard by reality and I feel like I spend half of my time denying them permission to add new self-hosted services to our stack.

Just last month a young padawan was pissed at the spent on an external auth service and had been pushing hard for a self hosted OSS solution which he was convinced he could handle by himself (which was most likely true, from a purely technical standpoint).

Since he wouldn't let it go, I "punished" him by having him spend one day in excel and powerpoint to prepare a cost benefit analysis to present to the architecture review board, including server cost, backups, redundancy, security, monitoring, pen-testing, auditing, his time and all the bells and whistles we needed to be compliant with all the ISO-x we have to be. (we're in a banking related field).

Our estimated internal cost ended up about 6x the one of the SASS solutions and still wasn't as reliable.

Most people don't understand the amount of effort it requires to run a secure & reliable system and if I had a dollar for everytime I heard it's as simple as "docker run", I could retire early.

[–] [email protected] 6 points 1 month ago* (last edited 1 month ago) (1 children)

Never upgrade to the latest and greatest of ... anything really, especially in production. Let others test it first, or as suggested already, have a staging environment where you test the upgrade first. I guess you can still downgrade nextcloud though, especially if you have a backup.

Are you using the AIO image? I don't know how well that works, but yeah, I absolutely hate automatic updates like that. I tried it once and I decided to use the plain "official but not supported" docker image instead, where I manage things myself. Never had an issue, and I can control which version I'm running, I can backup to wherever I want, using whichever system I want, etc.

[–] [email protected] 2 points 1 month ago

AIO has a updater but it is manual by default You need to enable automatic updates yourself, which.... Is done through a bash script you need to add yourself into the system crontab

And not only that, the instructions do say things could break and even suggests setting up backups for such

[–] [email protected] 2 points 1 month ago

You can still choose to installt he old version in NC30 and it will do so. and I upgraded to NC30 and my forms app continues to be functional. you can still give it a try.

[–] [email protected] 2 points 1 month ago (4 children)

Docker is kind of a giant mess in my experience. The trick to it is creating backup plans to recover your data when it fails. As such, I don't really recommend it to anyone at all.

[–] [email protected] 5 points 1 month ago

Docker is kind of a giant mess in my experience. The trick to it is creating backup plans to recover your data when it fails.

Thats the trick for any production service. Especially when you do an update.

[–] [email protected]:443 3 points 1 month ago

This is why you have a staging environment and backups for the very rare case where something goes wrong. I've only had to roll back a container maybe 3 times in 5 years at this point and can just roll back to a previous snapshot in a few minutes.

[–] [email protected] 3 points 1 month ago (1 children)

I wouldn't recommend Docker for a production environment either, but there are plenty of container-based solutions that use OCI compatible images just fine and they are very widely used in production. Having said that, plenty of people run docker images in a homelab setting and they work fine. I don't like running rootful containers under a system daemon, but calling it a giant mess doesn't seem fair in my experience.

load more comments (1 replies)
load more comments (1 replies)
load more comments
view more: next ›