116
submitted 1 week ago* (last edited 1 week ago) by AbnormalHumanBeing@lemmy.abnormalbeings.space to c/opensource@lemmy.ml
top 7 comments
sorted by: hot top controversial new old
[-] supersquirrel@sopuli.xyz 28 points 1 week ago
[-] BakedCatboy@lemmy.ml 27 points 1 week ago

We're seeing this at work too - our public git frontend is constantly getting scraped as well as our self hosted issue tracker. We had to spend days working on fail2ban and other kinds of tools to mitigate all the traffic that's adding tons of load to our instances, which otherwise would easily be able to handle the handful of employees who actually use these systems.

[-] Botzo@lemmy.world 22 points 1 week ago

Ah, yeah, I keep being scatterbrained and impulsive when crossposting, and forgetting he sadly doesn't even link direct links in the video description, instead of just to the website itself. Will add the link to the lemmy post bodies at least.

[-] floofloof@lemmy.ca 14 points 1 week ago

There are tarpits like Nepenthes but they use up your CPU resources and I imagine it would be pretty easy to update a scraper to recognize these generated pages, since they're all structurally similar.

[-] nomugisan@lemmy.dbzer0.com 12 points 1 week ago

I'm no Sysadmin but to me it sounds like we need a botnet/scraper resistant application layer protocol to replace HTTPS.

[-] haverholm@kbin.earth 8 points 1 week ago

So basically what Drew DeVault wrote about the other day I guess?

this post was submitted on 20 Mar 2025
116 points (97.5% liked)

Open Source

35295 readers
211 users here now

All about open source! Feel free to ask questions, and share news, and interesting stuff!

Useful Links

Rules

Related Communities

Community icon from opensource.org, but we are not affiliated with them.

founded 5 years ago
MODERATORS