39
submitted 5 months ago by neo@hexbear.net to c/technology@hexbear.net

Consider https://arstechnica.com/robots.txt or https://www.nytimes.com/robots.txt and how they block all the stupid AI models from being able to scrape for free.

you are viewing a single comment's thread
view the rest of the comments
[-] vk6flab@lemmy.radio 1 points 5 months ago
this post was submitted on 29 May 2024
39 points (100.0% liked)

technology

23313 readers
106 users here now

On the road to fully automated luxury gay space communism.

Spreading Linux propaganda since 2020

Rules:

founded 4 years ago
MODERATORS