12
submitted 1 year ago* (last edited 1 year ago) by doostee@programming.dev to c/programming@programming.dev

From a technical and legal standpoint, ignoring ethics and dignity, is there anything preventing us from scripting a scraper that recreates reddit posts in a lemmy instance? Like maybe top 50 posts of the top 20 subreddits, without comments. I think it would help convince people to join, since the major argument for sticking with reddit is that it has more content. Thoughts?

you are viewing a single comment's thread
view the rest of the comments
[-] pixelpop3@programming.dev 2 points 1 year ago

Leverage for what purpose? To fix reddit? Let reddit die or not die.

Reddit has always come after mirrors and they will easily get courts to take down the instances. Don't forget that prior to the API change they came after pushshift.

Additionally, anyone mirroring reddit on the moral basis that the content is owned by the creators and reddit is an exploitative rentseeker, has an obligation to not become a rentseeker themselves. This means things like ensuring that content that users voluntarily delete is also deleted in the mirrors. Reddit in fact had a large battle with pushshift about this years ago such that pushshift supposedly now only keeps history of moderator and admin edits. I agree with that ethically.

And in many cases you may be legally required to do this. To be clear Reddit made pushshift change to respecting user delete requests because of legal exposure and compliance risks.

Not to mention that you don't really know that anyone intends their content to be mirrored on sites they do not use. Particularly now that Reddit seems to be forcing private subreddits to be open. There's no moral high ground for doing this.

this post was submitted on 28 Jun 2023
12 points (64.3% liked)

Programming

17314 readers
17 users here now

Welcome to the main community in programming.dev! Feel free to post anything relating to programming here!

Cross posting is strongly encouraged in the instance. If you feel your post or another person's post makes sense in another community cross post into it.

Hope you enjoy the instance!

Rules

Rules

  • Follow the programming.dev instance rules
  • Keep content related to programming in some way
  • If you're posting long videos try to add in some form of tldr for those who don't want to watch videos

Wormhole

Follow the wormhole through a path of communities !webdev@programming.dev



founded 1 year ago
MODERATORS