14
submitted 1 year ago by kense@lmmy.dk to c/lemmy_support@lemmy.ml

Hi everyone

I am pretty new to all of this, so I'm currently in what I consider my "sandbox phase" of hosting a Lemmy instance.

Before promoting my instance to others I want to ensure that it's working as it should and federation is quick and "in sync" with the major instances.

Looking at the logs I see federation is at it and working at approx 4-8 req/s - but is there a way to speed this up?

Would more workers (and thus more cores on the server) in postgres speed things up? Or perhaps more parallel workers?

As stated I'm pretty new to this, and when searching for similar questions on lemmy/github I can't seem to find the answer to this specific question - it's usually implied that one should know all of this before hosting.

I've looked into how postgres is working, but am unsure if this is the only parameter that can be changed to speed up federation.

you are viewing a single comment's thread
view the rest of the comments
[-] chiisana@lemmy.chiisana.net 6 points 1 year ago

There is no way to “speed up” federation. Lemme doesn’t backfill comments, nor posts, and will only receive new content if at least one user on your instance subscribed to a specific community. This is because the federation process (beyond the initial search) is not a pull model in which your server requests information from other instances and stores it; but rather it operates on a push model in which you tell larger instances that you’d want new information (by means of having at least one user on your instance subscribe to the community) and they push that to you. Once that subscription is made, only new contents created afterwards are pushed to you, as fast as the other instance could. This as you can imagine puts a lot of stress on the larger servers as they’d need to update every interaction to thousands upon thousands of subscribed instances. “Seeding” content en mass (making dummy accounts to subscribe to everything) makes it worse because now the other instances are pushing entire community’s new updates to the new instance where no one ever touches it.

Instead. It’s probably best to just subscribe to a small handful of communities that you actually care about and intend to read, so only content you will consume gets pushed to you. Then invite like minded people to join and gradually, they too will add more that they’d actually read, so your instance isn’t just sucking the resources out of the fediverse at large but not actually using it.

[-] kense@lmmy.dk 1 points 1 year ago* (last edited 1 year ago)

I understand the idea of the "fediverse", but personally think that the "no backfill" is the biggest missing piece of the great puzzle.

If I, 2 years from now, wanted to create an instance on the ever-so-popular fediverse, why should I? I can't attract any new users, as they'll lack all of the historical posts that makes a community a community (the inside jokes, the posts we still remember, that sort of thing).

Where we are today, better load your instance up with content from the popular communities at the popular instances, to ensure you are able to attract new users in the future.

I get why that defeats some of the purpose of the fediverse, but as for "new instances" it is crucial to have content - or else you have no new users.

I've played around with the Lemmony script, and altered it to take only popular instances in account. Right now it only parameterize by active users pr community, but I've changed it to also look at users pr instance, so you can subscribe to e.g. top 10 communities of the top 50 instances, so you have the content on your instance for new users joining... But this is just me playing around (and causing the load on the popular instances) with the dream of some day being able to provide my users with the content they would also be able to view on the major instances.

[-] muddybulldog@mylemmy.win 3 points 1 year ago

I really hope you’ve considered how much storage you’re going to need to sustain this.

[-] kense@lmmy.dk 2 points 1 year ago

Hey, when life gives you lemons, buy more storage!

load more comments (2 replies)
load more comments (2 replies)
this post was submitted on 04 Aug 2023
14 points (100.0% liked)

Lemmy Support

4656 readers
15 users here now

Support / questions about Lemmy.

Matrix Space: #lemmy-space

founded 5 years ago
MODERATORS