35
submitted 2 weeks ago by dessalines@lemmy.ml to c/lemmy@lemmy.ml

I've recently added anubis to lemmy.ml, and it seems to be working well.

I have a PR to add anubis to lemmy-ansible (our main installation method), and I could use some help tweaking / optimizing its botPolicy.yaml config, for federated services.

Anyone with experience running anubis, this would be much appreciated.

you are viewing a single comment's thread
view the rest of the comments
[-] bjoern_tantau@swg-empire.de 1 points 2 weeks ago

Well behaved bots should see the origin in the headers and only crawl those.

this post was submitted on 09 Apr 2026
35 points (100.0% liked)

Lemmy

14583 readers
2 users here now

Everything about Lemmy; bugs, gripes, praises, and advocacy.

For discussion about the lemmy.ml instance, go to !meta@lemmy.ml.

founded 6 years ago
MODERATORS