158
submitted 2 weeks ago by yogthos@lemmy.ml to c/opensource@lemmy.ml
you are viewing a single comment's thread
view the rest of the comments

There's no excuse for this crap. Even if they insist on scraping every FOSS repo, there needs to be some logic to it (caches, diffs, longer intervals). These AI scrapers are so poorly thought out they are indistinguishable from DOS attacks.

[-] digdilem@lemmy.ml 28 points 2 weeks ago* (last edited 2 weeks ago)

You're not wrong.

Claudebot took down one of my sites repeatedly, hammering it for the same pages over and over at horrendous rates.

I ended up spending several days having to convert it to a SSG and hosted it on Cloudflare pages. A lot of work I didn't need to do.

this post was submitted on 25 Sep 2025
158 points (98.8% liked)

Open Source

41346 readers
114 users here now

All about open source! Feel free to ask questions, and share news, and interesting stuff!

Useful Links

Rules

Related Communities

Community icon from opensource.org, but we are not affiliated with them.

founded 6 years ago
MODERATORS