I think you're right. People will gravitate to the most stable large instances because their "All" will be as close to 100% as possible without doing anything special. I wrote a script to seed instances and update subscriptions, but it uses a single account that is subscribed to everything so that other users can see everything. That's not something that would normally happen. Maybe that needs to be part of the base software?

It'd be really cool if we had something similar to Mastodon's "Relays" where you basically subscribe to a firehose of posts from everyone whose on a server connected to that relay (they show up in the "Federated Timeline").

I don't know exactly how this would work for Lemmy, but it seems like if we had a system like this it could really help tackle this issue.

I think there is already some software that fetches content but it's early in development.

Correct. All also includes communities fetched but not subscribed to, however these are more like stubs. They are in your database but not being updated with activity since no one is subscribed. At least that’s my understanding.

There's no general answer, it depends on your personal preference.

If you want to have most content available, register on an instance which has an according policy; which federates with anybody and is federated by everybody (both directions can make a difference).

The downside however is, this also opens the door to all sorts of bad actors, including bots and spam.

So I personally tried to strike a balance and am so far quite happy on lemm.ee.

This tool is pretty handy to make informed decisions: https://fba.ryona.agency/ It allows you to check federation status both ways.

Well, it really comes down to how many subscriptions there are.

A small instance may only sub to 100 communities, so it is not too bad.

But on the flip side, it means that the big instancr needs to send everything to a huge number of small instances.

In practice I do not think it will be too bad, there will be a set of medium sized instances that most will be attracted to, and they will have the p80 of communities subbed. Smaller ones will be for more technical people who will not worry that they need to ensure the content is subbed to, as they will understand how it works.

I think over time, services that aggregate community details will spring up, and be incorporated into the lemmy search, so it is easier to find things across the entire fediverse, not just your instance. I think there will be a large set of muggle-type user improvements over the next couple of months.

Media takes up space. The text from posts and comments is trivial. The database for lemmy.world is only 25 GB. Wikipedia text is only 21 GB.

It's not quite as bad, because you're still being pushed what you subscribe to. So while you do indeed get a fair bit of content you might never see, it's necessary for you to be able to browse those communities and even being able to compute what threads are active/trending/hot/updated or whatever else filter you use. Because that's all computed locally on your instance.

It's also an efficiency advantage: if your instance has a lot of users, having everything locally means that you offer a much smoother experience, and also you're contributing to the remote instance not being so busy with traffic as you're not just proxying everything to it and increasing the remote's load.

For your storage concerns, there's nothing preventing you from purging content older than a week or two regularly via a cronjob.

It's not that bad so far:

8,0K    volumes/lemmy-ui
887M    volumes/pictrs
646M    volumes/postgres
1,5G    total