You are right. On the one hand, it's kind of bad, naive distributed architecture (my day job), it could have been done much better. On the other hand, the more important point is that it demonstrates an alternative to centralized. We'll learn a lot about usage patterns here, get new ideas, and either improve Lemmy or build something better from the ground up. Big thanks to Reddit for driving users this way to test scalability and get much better knowledge of usage.

This has definitely been a problem with communities being created on the bigger instances and not utilising smaller instances. Happy for someone to say I'm wrong etc, but I think there would be merit in capping instances to x number of users or communities, to force the user base to spread out.

Also, the way signups work, (ie you find a community you like then click sign up but that signs you up to that instance), further exacerbates the issue and the confusion around how federation works. The sign up links on each instance should lead either to a page with an instance finder, or to a random instance that matches the profile of, and is already federated with, the instance you were on. Otherwise the larger instances have a monopoly and are just going to lead to a bad user experience when they can't cope with the traffic.

It's a self defeating prophecy if users only want to sign up to the instances with the big communities, because then everyone is going to keep creating communities there and nobody is going to want to join a smaller instance.

I might be talking nonsense and am happy to be told why that is all wrong :)

A network of (“thousands of”) servers has — like most things — pros and cons.

Some of the pros are:

• The network is more resiliant against outages. If lemmy.ml is down, all other users can still access the network.
• It's hard to take legal action against the network or to buy it out (like Big Players™ like to do to get rid of potential competitors).
• It allows various similar or even conflicting moderation policies. The network, i.e. the infrastructure doesn't allow or prohibit any specific opinion (the communities do).
• It allows for different ways to pay the bills: goodwill of the admin, donaitions, ads, fee or selfhosting. The latter also allows great control over the data so you control your privacy.

Some of the cons are:

• Content is replicated across servers, which increases the total amount of data stored.
• Latency and speed suffer.
• Interoperability with the wider Fediverse is less than 100%, which can create confusion and frustration.
• Discovery is more difficult.

Since Lemmy instance are not backed by commercial interest, but rather by nice volunteers and donors that have money and time to spare, they will be heavily affected by economic downturns (we still can see commercial interests still affect users negatively tho with reddit). Here are my thoughts on the matter:

• as far as I understand the owner of the domain: https://lemmy.world even has to pay for this fancy domain name in the DNS system ... every month subscription service style
  • (and tbh I hate the Domain name system) why should I fund it with my own money?
  • if you hosted with an onion site over tor that expenditure would not exist, but how would users discover your site then? Let me know if you know something about this
• in times of deflation (meaning money becomes worth more, spending some money on a self hosted lemmy instance becomes nonsensical)
• tbh if I hosted a lemmy instance and the users of my instance posted high quality content in quantity I would use it to train my own LLM, that would at least create some economic incentive for me to host such a page ... but managing spam and bots will be HARD

That is why you should always back up your comments on your personal device, would be nice if lemmy had an automated way of doing this (I should look into this more)

I have my own Lemmy instance running on my home server, but I'm here. "But Bizzle," you may be asking yourself, "why go through all the trouble of configuring your own instance just to wind up on Lemmy.World anyway?"

I'm glad you asked! And the answer is that federation only fetches parent comments. I'm glad Lemmy exists, and I'm going to keep using it, but we need federated sibling comments for this to actually be good, in my opinion.

EDIT: I actually couldn't have been more wrong.

I'm quite worried of how well this federation system will work in the long run, especially when more people coming from Rexxit. As people make more post/comments, every federated instance will have to cache more redundant contents from each other, which also will use more storage thus increasing the fee of every instance hoster. There's also another problem of visibility in search engines. Because Lemmy/Kbin can be hosted by anyone, it makes searching on a specific domain impossible, unlike how I can just add "reddit" in the search query. Also since there are multiple Lemmy/Kbin instances, there's a chance there'll be similar communities spread over, fragmenting the communities even further. Until they can find a way to fix those problem, I don't think federation is suited for large scale communities.

As for fragmentation problem, maybe adding a global search for communities like this will help reducing fragmentation. Users can still make their own community in their instance, while other people who don't need to can easily find the community they want.

I've suggested a routing protocol to the lemmy devs - to use federated instances to route all the messages to other federated instances. The idea was received with some interest, but it seems that people believe that there's still a ton of performance that can be squeezed out from the current architecture through optimisations.

I just commented on this in another thread: https://lemmy.world/comment/76011

TL;DR: The server-to-client interactions on Lemmy are a lot heavier than the server-to-server interactions, so even if you're just using your own server to interact with communities on other servers, it should still take load off of the servers you would have been using directly.

It's a bit worse than that actually. I'm now seeing several communities with exactly the same name that originate on different servers - so clearly Lemmy doesn't have a rule about duplication once you cross a server boundary. That's going to get unwieldy quite fast particularly if, I dunno, "Aww" gets popular on two separate servers at the same time - I guess I'll have to subscribe to both...

Every server just has a cache … there is no profit for the whole network …

I wouldn't say that caching is no profit. Yesterday there were several times when lemmy.ml was struggling or effectively down for some people, but despite complaints over there I could read lemmy.ml communities just fine through my instance. Caching meant that I was isolated from the service interruption, and the lemmy.ml server was isolated from my contribution to its load.