35
submitted 2 weeks ago by dessalines@lemmy.ml to c/lemmy@lemmy.ml

I've recently added anubis to lemmy.ml, and it seems to be working well.

I have a PR to add anubis to lemmy-ansible (our main installation method), and I could use some help tweaking / optimizing its botPolicy.yaml config, for federated services.

Anyone with experience running anubis, this would be much appreciated.

you are viewing a single comment's thread
view the rest of the comments
[-] adam@doomscroll.n8e.dev 5 points 2 weeks ago

It would be nice to have a much more aggressive anti-bot stance for communities/content that aren't local. If google or any other crawler wants to crawl c/lemmy@lemmy.ml then it should do it on the source instance. Doing it on mine makes no sense.

[-] bjoern_tantau@swg-empire.de 1 points 2 weeks ago

Well behaved bots should see the origin in the headers and only crawl those.

this post was submitted on 09 Apr 2026
35 points (100.0% liked)

Lemmy

14580 readers
11 users here now

Everything about Lemmy; bugs, gripes, praises, and advocacy.

For discussion about the lemmy.ml instance, go to !meta@lemmy.ml.

founded 6 years ago
MODERATORS