Lemmy Server Performance

420 readers

1 users here now

lemmy_server uses the Diesel ORM that automatically generates SQL statements. There are serious performance problems in June and July 2023 preventing Lemmy from scaling. Topics include caching, PostgreSQL extensions for troubleshooting, Client/Server Code/SQL Data/server operator apps/sever operator API (performance and storage monitoring), etc.

founded 1 year ago

MODERATORS

[email protected]

lemmy_server API for Clients and Federation alike, concurrency self-awareness, load sheding, and self-tuning (lemmy.ml)

submitted 1 year ago by [email protected] to c/[email protected]

3 comments fedilink hide all child comments

Heavy loads have been brought up several times, major social events where people flock to social media. Such as a terrorist bombing, submarine sinking, earthquake, nuclear meltdown, celebrity airplane crash, etc.

Low-budget hosting to compete with the tech giants is also a general concern for the project. Trying not to have a server that uses tons of expensive resources during some peak usage.

Load shedding and self-tuning within the code and for SysOps to have some idea if their Lemmy server is nearing overload thresholds.

See comments:

you are viewing a single comment's thread
view the rest of the comments

[–] [email protected] 2 points 1 year ago

10 second http timeout

https://github.com/LemmyNet/lemmy/blob/f5209fffc1de527db7ea007d463c158b36fda515/src/lib.rs#L39

I almost worry this default 10 second timeout is too low, and that is actually causing federation retries to flock between major peer servers. But this is the kind of self-tuning within the app that self-awareness of concurrency could adjust.

Areas I know of that Lemmy does HTTP:

Federation replication of content to peers
When making a Post on Lemmy, a url posting fetches a preview of the content

I'm not even sure where that value is used, any help on delving into this is appreciated.