13
Boo! the upstream crawler is still borked (up a tiny bit to 9k communities now).
Lemmy seems to be on the fritz lately. I've noticed lots of weird federation/sync issues, with even the big places like .world having problems recently. Very strange behavior across the board the last couple weeks.
There's been a recurring federation delay between lemmy.ml and lemmy.world:
https://grafana.lem.rocks/d/cdfzs0dwal3pca/federation-health-time-behind?orgId=1&var-instance=lemmy.ml&var-remote_software=lemmy&from=now-30d&to=now&var-remote_instance=lemmy.world&timezone=browser
The LW admins are aware, but I don't think they've yet nailed down the cause (it fixed itself yesterday, but has started again today). The same thing might be stopping the crawler from getting a full API response from them.
I have a test instance that's subscribed to a couple of communities, but is quiet enough to be able to monitor federation, and if you do that, you realise that there are activities that are meant to happen, and just flat-out don't. Other instances can recover (if they receive a reply for a post they don't know about because LW forgot to Announce it, then they can fetch it instead), but it's all a bit flaky, yeah.
I'm having similar issues with a number of instances. Users have reported verified federation delays between walledgarden and several other places including lemmy.ml, hexbear.net, feddit.org, vegantheoryclub.org, and possibly others. Before anyone says it- yes I know they don't federate with everyone but walledgarden isn't being blocked/defedded.
Walledgarden seems to be able to retrieve the data from them, but WG communities stopped federating to them a few days to a week or more ago in many cases, and as recently as yesterday in others. I've been working through it with our host but so far we haven't figured it out since there isn't anything obvious causing the problem on our side. As far as I can tell replies and interactions from walledgarden still make it to those instances without issue.
It's very frustrating.
I mentioned the other day that requests to walledgarden error (curl says 'transferred a partial file'). If another instance doesn't get a good response for these requests, then after 3 days, it'll mark the other instance as dead, and not federate out to it. Once this happens, it's a pain, because traffic from walledgarden doesn't make them realise that it's actually alive, you have to wait until it tries again and gets a successful response.
If you haven't done so already, it might be an idea to use curl to query your site, and see what the response in nginx says.
Update to this-
After talking with my host some adjustments were made that should correct the curl error. I still don't believe this has anything to do with the lost sync but it's one less thing to worry about. Thanks for the information and help!
I saw your comment but I was mainly concerned with getting the instance to show up on lemmyverse at the time, which I suspected was the connection being blocked by our host (it was). Coincidentally since getting listed lemmyverse has been having its own problems so I put it on the back burner.
I'll do some deeper digging into this and see what I can figure out. I don't think it has anything to do with the communities falling out of sync since it's never mattered before now and WG isn't the sole instance being impacted, but it should work correctly either way.
Yeah Lemmy.vg currently seems to be isolated from the rest of Lemmy for some reason.
this post was submitted on 21 Nov 2024
13 points (100.0% liked)
Trending Communities
8 readers
34 users here now
Bot-driven site intended to highlight the many Communities that are trending all across the Fediverse.
Daily data is from Lemmyverse.
Posts are limited to the bot, but any comments are read by a human, so please leave one if you have any suggestions or complaints.
Publicise your communities:
Community Promo
New Communities
Other discovery tools:
ponder.cat
Filtered Instances
enterprise.lemmy.ml
voyager.lemmy.ml
ds9.lemmy.ml
alien.top
mujico.org
rqd2.net
zerobytes.monster
lemmygrad.ml
hilariouschaos.com
Filtered Communities
trendingcommunities@feddit.nl
trendingcommunities@lemmynsfw.com
test@lemm.ee
test@lemmy.ml
main@feddit.org
main@lemmy.ca
lemmyworld@lemmy.world
home@lemmy.zip
div0@lemmy.dbzer0.com
announcements@lemmy.ml
moderators@lemmy.world
meta@programming.dev
test@feddit.org
announcements@vegantheoryclub.org
meta@lemm.ee
sdfpubnix@lemmy.sdf.org
main@hexbear.net
founded 1 year ago
MODERATORS