171
Announcing the Data.gov Archive (lil.law.harvard.edu)
submitted 1 week ago by misk@sopuli.xyz to c/technology@lemmy.world
top 17 comments
sorted by: hot top controversial new old
[-] Bakkoda@sh.itjust.works 34 points 1 week ago

I'll officially be seeding more of this type of stuff (annas, cdc, etc) than all of my "Linux isos". Feels good.

[-] Tikiporch@lemmy.world 5 points 1 week ago

Did they create a torrent for this, or just straight downloads?

[-] finicky_foyer@sh.itjust.works 5 points 1 week ago

It appears to be hosted via S3 compatible object store. Their docs provide examples to browse/download via awscli or boto3 (as well as rclone).

https://source.coop/repositories/harvard-lil/gov-data/description https://source.coop/repositories/harvard-lil/gov-data/access

[-] errer@lemmy.world 28 points 1 week ago

16TB is within the capacity of individuals. Given I don’t trust even Harvard to be safe with the impending onslaught, anyone with the means of downloading a copy for themselves should do so.

[-] Xanza@lemm.ee 6 points 1 week ago

Cold storage is $2 per TB/mo. Could easily be crowdfunded. $410/yr to keep it around.

[-] just_another_person@lemmy.world 24 points 1 week ago

At least private institutions are stepping up. Where's the rest of the universities?

[-] NaibofTabr@infosec.pub 16 points 1 week ago

Dependent on federal money.

[-] corsicanguppy@lemmy.ca 5 points 1 week ago

... Thus, not saying a thing if they do actually cache it.

[-] rbos@lemmy.ca 12 points 1 week ago

I work for a Canadian university library and I assure you that data archival efforts are proceeding apace. I can't really say more without possibly endangering American colleagues.

[-] tastysaganaki@reddthat.com 5 points 1 week ago
[-] NudeNewt@lemm.ee 3 points 1 week ago

Remember folks, to avoid Linux ISO honeypots you must enable Super-Seeding.

[-] Xanza@lemm.ee 2 points 1 week ago

Is there a list of super-seeding complaint clients? I can't seem to find one. Hell, I can't even check to see if my client supports it.

[-] NudeNewt@lemm.ee 1 points 1 week ago

Most p2p clients support it, you'll need to refer to their documentation to enable.

For the most part all you need to do to enable it is right click a completed download then select "super seeding" mode. It's at least that straightforward for QBT.

[-] Xanza@lemm.ee 2 points 1 week ago

I never knew. My version of Deluge is 1.3.5, and super seeding requires 2.0+. So that looks like the issue. lol

[-] NudeNewt@lemm.ee 2 points 1 week ago

Hmm, in that case I recommend you read into "Bandwidth Tweaking":

https://deluge-torrent.org/userguide/bandwidthtweaking/

[-] Xanza@lemm.ee 2 points 1 week ago

I'm gonna look into upgrading, but I've implemented this as well. Thanks.

[-] tonytins@pawb.social 2 points 1 week ago

Thank goodness. I was hoping for something like this.

this post was submitted on 09 Feb 2025
171 points (100.0% liked)

Technology

63023 readers
1451 users here now

This is a most excellent place for technology news and articles.


Our Rules


  1. Follow the lemmy.world rules.
  2. Only tech related content.
  3. Be excellent to each other!
  4. Mod approved content bots can post up to 10 articles per day.
  5. Threads asking for personal tech support may be deleted.
  6. Politics threads may be removed.
  7. No memes allowed as posts, OK to post as comments.
  8. Only approved bots from the list below, to ask if your bot can be added please contact us.
  9. Check for duplicates before posting, duplicates may be removed
  10. Accounts 7 days and younger will have their posts automatically removed.

Approved Bots


founded 2 years ago
MODERATORS