20
submitted 12 hours ago by moondog@hexbear.net to c/technology@hexbear.net

If I want to download all of this person's artwork https://x.com/CaoAmoss/media for use in my ttrpg games , how can I do that easily? I've been using gallery-dl a lot lately and it works great for artstation, deviantart, tumblr, etc. but twitter is a nightmare to scrape, at least with gallery-dl, and their scraper for nitter.net seems to be broken.

top 10 comments
sorted by: hot top controversial new old
[-] Xiisadaddy@lemmygrad.ml 8 points 12 hours ago

Have you considered sending that person a message and saying "Hey I'd like to use your artwork for a personal project of mine would you possibly be willing to provide me with a zip file containing anything youre willing to share?" or something like that.

[-] moondog@hexbear.net 4 points 12 hours ago

I hadn't considered that, no. Practically I think it'd be more effort for both me and the artist to do that than for me to just download all of their artworks with some tool. If you're asking because you think I should get their consent for using it, I would totally agree if it was a publicly broadcast game or something that their art was being used in, but just in a game between my friends I don't think it's necessary.

[-] Xiisadaddy@lemmygrad.ml 9 points 11 hours ago

So i do think its good to get consent but i didnt suggest that for this reason. If someones posting specific things on twitter they know theyll be scraped and are kind of implicitly allowing those specific works to be used for free by people.

This though would potentially allow you to get access to more stuff that isnt posted publicly, and would also allow the artist to know someone likes their work enough to do this. Maybe they'd even want to play the game with you.

As for difficulty most artists have their works saved and all they'd have to do is share a google drive folder or something and have you download it. Super easy.

Also its just like... talking to people is good yknow? I mean this is someone whose work you like, and the internet makes things so impersonal already. Starting a discussion, and getting to know someone with similar tastes just seems like a net positive. A wasted opportunity for a potential friend if you dont atleast try imo. But ultimately its up to you what you do.

[-] moondog@hexbear.net 3 points 8 hours ago* (last edited 8 hours ago)

That's a good point. I think I will give them a message later. Thanks for the advice!

[-] FloridaBoi@hexbear.net 4 points 12 hours ago

What if they point you to a different site with their art that is easier to scrape?

[-] Orcocracy@hexbear.net 3 points 11 hours ago

They might appreciate hearing from someone who likes their work.

[-] KnilAdlez@hexbear.net 2 points 12 hours ago

I have done it with selenium but it was a pain in the ass

[-] Red_sun_in_the_sky@hexbear.net 1 points 10 hours ago

Ok if gallery dl didn't work. You should try either bulk image downloader or jdownloader. I think both can grab large number of images in a given link.

[-] ozymandias@lemmy.dbzer0.com 1 points 11 hours ago

generally, i would start with this:
https://www.google.com/search?q=scrape+twitter+github
and probably find some python project that’s been updated recently

[-] cbd@hexbear.net 1 points 11 hours ago

If you host an instance of nitter locally, you can scrape it. There's a package that I'll link when I get a chance. However, this possibly puts the Twitter account you use at risk of a ban, haven't tried it enough to find out.

this post was submitted on 17 Aug 2025
20 points (100.0% liked)

technology

23915 readers
242 users here now

On the road to fully automated luxury gay space communism.

Spreading Linux propaganda since 2020

Rules:

founded 5 years ago
MODERATORS