this post was submitted on 21 Mar 2025
213 points (99.1% liked)

Linux

6821 readers
252 users here now

A community for everything relating to the GNU/Linux operating system

Also check out:

Original icon base courtesy of lewing@isc.tamu.edu and The GIMP

founded 2 years ago
MODERATORS
 

LLM scrapers are taking down FOSS projects' infrastructure, and it's getting worse.

you are viewing a single comment's thread
view the rest of the comments
[โ€“] sudo 2 points 5 days ago (1 children)

I've been criticizing Anubis and Proof of Work solutions in general. Its my speculation that they mostly work just by requiring you to execute javascript not by being an actual burden on the bots CPU.

I already stopped visiting 3 websites I used to frequent because I suddenly got redirected to Anubis and was told to enable JS. All my browsers have JS disabled by default and are configured to never keep any local storage/cookies out of principle. I won't change that just to get to a website that is mostly static HTML. Even on the occasion where I bite the bullet it takes between 5 - 30 seconds to complete the PoW challenge on my main machine. I hope this trash doesn't get popular. I really doubt that this makes much difference to more sophisticated crawlers that run on enterprise hardware.