Thank you Ruud for hosting! Your work is much appreciated.
this post was submitted on 12 Jun 2023
333 points (99.1% liked)
Lemmy.World Announcements
28381 readers
3 users here now
This Community is intended for posts about the Lemmy.world server by the admins.
Follow us for server news π
Outages π₯
https://status.lemmy.world
For support with issues at Lemmy.world, go to the Lemmy.world Support community.
Support e-mail
Any support requests are best sent to [email protected] e-mail.
Report contact
- DM https://lemmy.world/u/lwreport
- Email [email protected] (PGP Supported)
Donations π
If you would like to make a donation to support the cost of running this platform, please do so at the following donation URLs.
If you can, please use / switch to Ko-Fi, it has the lowest fees for us
Join the team
founded 1 year ago
MODERATORS
Thanks for hosting this great instance! We appreciate all you're doing!
2 restarts done already :-)
Hmm. I guess the delay in posting is not related to nginx. I now have the same conf as a server that doesn't have this issue.
I'm only familiar with the high-level Lemmy architecture, but could it be related to database indices being rebuilt?
Thanks for putting in the time to make this run smoothly
Godspeed to you over the coming days man. Really appreciate you putting this together and the extra work it takes when tackling something like this (both being new to the platform and the tech still being in relative infancy) - not to mention the crazy scaling happening. I will definitely be pitching in to help make sure the server stays up!!
Good luck today lol
Thanks for your work on this server!
Keep up the good work!
I joined this instance and also mastodon.social, first time using the Fediverse and as excited as confused lol
Thank you so much for this amazing instance!
You're welcome!
Hehe, the joys of troubleshooting and profiling. Isn't it fun?
Hmm if it takes too long the fun disappears... ;-)
You got this. <3
I don't have experience scaling Lemmy, but I do have experience scaling stuff in general. I'm sure you've got a few people here who'd be willing to talk things through with you if you get too frustrated.
And don't forget to breathe and step back if you have to. Your well being is more important.
@[email protected] have you seen https://lemmy.world/post/72136 ?
Thanks! Hadn't seen, but changed that (also to 512!) yesterday.
That is definitely a good lead
Thanks a bunch! I'll be donating for what it's worth. I really like it here.
Something is weird.
I opened this post from main page "subscribed listing", but the title showed "I can't find any cannabis cultivation community", but the comments were same. I initially thought I have opened a wrong post, but the comments were mentioning "Good work Ruud", so I refreshed and it fixed post's title.
Have you noticed the issue?
I've noticed a couple oddities as well.
- I refresh a page and a completely different page loads instead
- An autorefresh hits the community tab, but it loads up 10 posts from a single community I'm sure it'll get sorted out eventually lol
It's happened to me a few times as well (not just on this instance, think it's a bug in Lemmy itself). So far I'e not found a reproducible pattern though so it's a tricky one to bug report effectively.
Yeah, I tried opening it again a few times, no luck yet. Will see if I can figure out any pattern.
I had something similar happen yesterday.
I opened a thread about pokemon, browsed it for a bit, did some stuff in other tabs, and clicked back to the pokemon tab maybe an hour later to browse some more.
The post had changed to one where a user was asking for relaxing game recommendations and it was loading in new comments that seemed to be from that post, but I could still see the comments that had already loaded from the pokemon post when I scrolled down.
When I refreshed it changed back to the pokemon post and only showed comments from that.
Since I have you here, if I start my own instance do I absolutely have to use docker? I've never had good experiences with it and would rather just install programs the old-fashioned way
Well if they can create a docker image out of it, you should be able to install it on a VM.. but I run it in Docker because it makes everything so easy manageable...
It is possible to do it without docker... but nobody recommends it :)
Docker is not necessary, lemm.ee for example is running without docker!
Here is documentation for setting it up: https://join-lemmy.org/docs/en/administration/from_scratch.html
Of course you can fully adapt it to your own use case. The Lemmy backend is a single binary, you don't even need to build it on the same machine which will run it. There's no hard requirement to use nginx or anything like that either - if you understand what this guide is doing, you can replace all the unimportant parts as needed.
load more comments
(2 replies)
There is a how-to on how to set up your own instance without docker using ansible: https://join-lemmy.org/docs/en/administration/install_ansible.html
Note that this is just basically a script to deploy lemmy on a remote server. And it uses docker. It just does it for you. (Mostly)
Oh, oof. Didn't look into it much further as the docker solution would have suited me best also. Thanks for the heads-up
load more comments
(1 replies)
load more comments
(1 replies)
Any progress on this. I've been thinking about it too. Couple of ideas:
Too many indexes needing to update when an insert occurs?
Are there any triggers running upon insert?
Unlikely but there isn't a disk write bottleneck? Might be worth running some benchmarks from the VM shell.
I was thinking that as well, itβs like the post gets βcheckedβ or something like that and that gives a timeout of 20secs. It could be an api or database but somehow my spidey sense says this could well be in code. Some extra calls to filter things maybe? Using an external server? Or even the propagation to the others? (Idk how this federation thing connects to the others, could be just that; maybe another server that is the bottleneck) I just found the 20 seconds suspicious given that is the default timeout
load more comments
(1 replies)
load more comments
(1 replies)
Hey. From my own experience - Nginx is awesome and fast when it is working, but the more you want from it, the more difficult it becomes.
Give Caddy a try. This reverse proxy has always been excellent for me. It has HTTP3 (QUIC) support, automatic ACME and overall excellent configuration in terms of simplicity and user friendliness.
Caddy is not a good choice if you need TCP/UDP proxy. It's only HTTP/HTTPS proxy.
load more comments
(6 replies)
Good luck with it.
Great work Ruud!
Thank you, Ruud!
Thanks for hosting.
Just joined but nice to see work happening to keep it working :)
Somehow I donβt think the slowness when posting or saving is due to the nginx server / reverse proxy running inside the container.
I would still think itβs related to inserts and updates in the DB, but I havenβt had time to look into it on my instance, sorry!
Edit wait! Posting and saving is fast now! What did you change? Nicely done! π
No it's not! Not for me anyway. Yes I'll be looking into that, but first migrate the server!
load more comments
(1 replies)
load more comments
(2 replies)
Well worth any inconvenience, thank you so much for hosting!
view more: next βΊ