Lemmy Server Performance

420 readers

1 users here now

lemmy_server uses the Diesel ORM that automatically generates SQL statements. There are serious performance problems in June and July 2023 preventing Lemmy from scaling. Topics include caching, PostgreSQL extensions for troubleshooting, Client/Server Code/SQL Data/server operator apps/sever operator API (performance and storage monitoring), etc.

founded 1 year ago

MODERATORS

[email protected]

Lemmy Server Performance, two birds with one stone, mass deletes of content: Account Delete, Community Delete, User Ban - moving these operations to a linear queue and add undo support / grace period (lemmy.ml)

submitted 1 year ago* (last edited 1 year ago) by [email protected] to c/[email protected]

10 comments fedilink hide all child comments

A general description of the proposed change and reasoning behind it is on GitHub: https://github.com/LemmyNet/lemmy/issues/3697

Linear execution of these massive changes to votes/comments/posts with concurrency awareness. Also adds a layer of social awareness, the impact on a community when a bunch of content is black-holed.

An entire site federation delete / dead server - also would fall under this umbrella of mass data change with a potential for new content ownership/etc.

you are viewing a single comment's thread
view the rest of the comments

[–] [email protected] 2 points 1 year ago (1 children)

This is weird. Account deletion should be handled by JOIN at lookup time, so comments/posts only display if the account is active. No mass updates, pipelines or otherwise

[–] [email protected] 2 points 1 year ago (1 children)

AccountDelete has a marketed feature to overwrite all post and comment content.

[–] [email protected] 2 points 1 year ago

Hmm ok, false sense of security there since another advertised feature is the open API (meaning no restrictions on scraping bots so there will definitely be archives of deleted posts), but whatever.

How does this sound: encrypt the comments in the db using a random key stored in the account row. Then at account deletion, overwrite that key, so the comments can no longer be decrypted. Maybe there is a way to purge those comments altogether during the next VACUUM. No idea how often that happens though.