389
you are viewing a single comment's thread
view the rest of the comments
[-] nutomic@lemmy.ml 11 points 1 week ago

Cache size is limited and can usually only hold a limited number of most recently viewed pages. But these bots go through every single page on the website, even old ones that are never viewed by users. As they only send one request per page, caching doesnt really help.

[-] jagged_circle@feddit.nl -3 points 1 week ago

Cache size is definitely not an issue, especially for these companies using cloudflare

[-] nutomic@lemmy.ml 6 points 1 week ago

It is an issue for the open source projects discussed in the article.

this post was submitted on 20 Mar 2025
389 points (99.7% liked)

Open Source

35359 readers
226 users here now

All about open source! Feel free to ask questions, and share news, and interesting stuff!

Useful Links

Rules

Related Communities

Community icon from opensource.org, but we are not affiliated with them.

founded 5 years ago
MODERATORS