This arises from the good ol issue of everybody just migrating to the same three or four big servers which end overloaded with their own users and can't send updates to other instances.
I remember the same happening to Mastodon during the first few exodus until a combination of people not staying, stronger servers and software improvements settled the issue.
I can barely get updates from lemmy.ml and lemmy.world isn't much better
Beehaw seems to perform okey.
I seriously thought I'm alone with this issue, but it seems it's fairly common for people hosting on their own. Same as you guys, it won't sync everything, some communities are even "stuck" with posts from a day back, even though there were many new ones posted.
Kind of off topic question, but I guess it's related? Is there anyone that can't pull a certain community from an instance? I seem to can't pull !asklemmy@lemmy.world or anything from that community, that includes posts and comments. No matter how many times I try, it won't populate on my instance.
EDIT: Caught this in my logs:
lemmy | 2023-06-20T08:48:21.353798Z ERROR HTTP request{http.method=GET http.scheme="https" http.host=versalife.duckdns.org http.target=/api/v3/ws otel.kind="server" request_id=cf48b226-cba2-434a-8011-12388c351a7c http.status_code=101 otel.status_code="OK"}: lemmy_server::api_routes_websocket: couldnt_find_object: Failed to resolve actor for asklemmy@lemmy.world
EDIT2: Apparently it's a known issue with !asklemmy@lemmy.world, and a bug to be fixed in a future release.
Does your server have enough power and workers to handle all the federated messages? Or is it constantly at 100% CPU?
The machine is a dedicated server with 6 cores, 12 threads, all of which are usually under 10% utilization. Load averages currently are 0.35 0.5 0.6. Maybe I need to add more workers? There should be plenty of raw power to handle it.
Yeah that sounds about enough to handle the load. How many workers do you use? And do you see any errors in your logs about handling messages? You could try to search for that particular thread to see if all replies are handled correctly?
Update: Did a -f watch of the logs for WARN messages while upping worker counts. Seems 1024 was the sweet spot. Upped further to 1500, and the warnings for expired headers have entirely stopped in large part. So it seems this was the solution.
Thanks for your help!
A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.
Rules:
Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.
No spam posting.
Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it's not obvious why your post topic revolves around selfhosting, please include details to make it clear.
Don't duplicate the full text of your blog or github here. Just post the link for folks to click.
Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).
No trolling.
Resources:
Any issues on the community? Report it using the report flag.
Questions? DM the mods!