this post was submitted on 21 Jun 2023

26 points (100.0% liked)

/0

1549 readers

74 users here now

Meta community. Discuss about this lemmy instance or lemmy in general.

Service Uptime view

founded 1 year ago

MODERATORS

[email protected]

Introducing The Lemmy Overseer (overseer.dbzer0.com)

submitted 1 year ago by [email protected] to c/[email protected]

14 comments fedilink hide all child comments

Hey everyone, you may have noticed that some of us have been raising alarms about the amount of spam accounts being created on insufficiently protected instances.

As I wanted to get ahead of this before we're shoulders deep in spam, I developed a small service which can be used to parse the Lemmy Fediverse Observer and retrieve instances which are suspicious enough to block.

The Overseer provides fully documented REST API which you can use to retrieve the instances in 3 different formats. One with all the info, one with just the names, and one as a csv you can copy-paste into your defederation setting. You can even adjust the level of suspicion you want to have.

Not only that, I also developed a python script which you can edit and run and it will automatically update your defederation list. You can set that baby to run on a daily schedule and it will take care that any new suspicious instances are also caught and any servers that cleared up their spam accounts will be recovered.

I plan to improve this service further. Feel free to send me ideas and PRs.

top 13 comments

sorted by: hot top controversial new old

[–] [email protected] 7 points 1 year ago (1 children)

I added Dockerfiles to make containers of them for those running docker-compose Lemmy instances.

https://github.com/db0/lemmy-overseer/pull/1

[–] [email protected] 7 points 1 year ago (1 children)

Cheers mate, I'm in bed atm so I'll check tomorrow

[–] [email protected] 1 points 1 year ago (1 children)

I got the client to run in a container on a cronjob docker run --rm --name overseer-eye -e LEMMY_DOMAIN=discuss.online -e USERNAME=overseer -e PASSWORD=<PASSWORD> lemmy-overseer-eye:latest

I thought I'd containerize the server too, but it lacks a schema. Or I'm missing something?

[–] [email protected] 1 points 1 year ago (1 children)

The eeever is just a python script. Not sure what schema you need

[–] [email protected] 0 points 1 year ago (1 children)

The server has a database connection to sqlite or postgres.

[–] [email protected] 1 points 1 year ago (1 children)

It's setup to have one yes, but I haven't created a db yet. It's my next step

[–] [email protected] 1 points 1 year ago

Well, that explains it then. Thanks. I thought I was going crazy. Sorry about that.

[–] [email protected] 5 points 1 year ago (1 children)

This is awesome, would you be open to contributions outside the realm of spam prevention?

I've been working on some scripts/apis to pre-search communities from federated instances so they immediately show up within an instance's search capability. Was also thinking about doing a bot account to auto subscribe.

Seems like this could fit with the whole overseer/curation theme. Would you agree or nah?

[–] [email protected] 4 points 1 year ago

I'm always up for collaboration and prs. I'm adding now some cool new capabilities so once I have that you'll be able to build much easier on it

[–] [email protected] 1 points 1 year ago (1 children)

This sounds cool! Similar vein what I have been trying to do on Lemmyverse - Determine trust of federated instances based on trust lists from other instances, along with instance stats over time. I'm starting to collect stats like Instance User count by Time, so I can potentially export lists of instances with a given score for each Instance, which could be automatically read by your defederation scripts.

I like your threshold:

# If there's this many registered users per local post+comments, this site will be considered suspicious
ACTIVITY_SUSPICION = 20

I may steal some of your code to build out my scoring algo too :)

[–] [email protected] 2 points 1 year ago (1 children)

Feel free to Colab on the overseer directly too

[–] [email protected] 1 points 1 year ago

I'll have a look through it tonight. 👌🔥🔥

[–] [email protected] 1 points 1 year ago* (last edited 1 year ago)

Cool. I was wondering when someone would implement this sort of email RBL list.

I'm not finding any info on The Overseer how instances are deemed suspicious and what mechanisms there are for reporting/disputing. How are instances scored?

load more comments