I have posts and comments on Hexbear from 4 hrs ago that don’t show up
I have a comment on Lemmygrad from 4 hrs ago that doesn’t show up
A recent series of posts from ml to other instances have almost no interaction locally. When viewed remotely, they have higher vote counts (no more comments to compare).
I haven’t extensively looked at the new feed to see if posts are outright missing, but I see the same general pattern spot checking posts that do appear: vote totals much lower than elsewhere, missing comments
I don’t see issues reported on https://phiresky.github.io/lemmy-federation-state/site
Yeah I noticed a big slowdown starting a few hours ago, but I didn’t try to identify specifics. Over on .world there are lots more newer posts.
I assume world is more broadly federated, but could be wrong. There’s definitely something specifically up with ml though.
I’m used to world and ml having about the same stuff, at least with regard to my own subscriptions. Ml actually has a few rederations like hexbear that are missing from world, and I read hexbear’s covid group.
@nutomic@lemmy.ml @dessalines@lemmy.ml tagging because something’s broke
Even this post has a comment that didn’t federate from Hexbear, it’ll be 2 soon enough: https://hexbear.net/comment/7223117
It doesn’t show anything different when I reverse them, but the problem isn’t outgoing it’s incoming.
3 upvotes, zero comments: https://lemmy.ml/post/48485322?scrollToComments=true
18 upvotes, 16 comments: https://hexbear.net/post/8719960
Same post.
Or, to the extent it’s also
incomingoutgoing, I haven’t tried to diagnose that.That means that something is going on with hexbear’s outgoing federation. You’ll need to ask them if there’s something going wrong.
Plenty of other folks are chiming in with the same experiences. It’s very clearly not just Hexbear. Is every other instance broken? Why are they federating just fine with each other, then?
https://lemmy.world/post/47958962
https://lemmy.ml/post/48512217
https://discuss.online/post/40968502
https://lemmy.ml/post/48508670
https://aussie.zone/post/33509821
https://lemmy.ml/post/48507829
https://piefed.blahaj.zone/c/womensstuff/p/792979/thoughts-on-this
https://lemmy.ml/post/48509886
https://startrek.website/post/40652133
https://lemmy.ml/post/48504964
https://lemmy.dbzer0.com/post/70286030
https://lemmy.ml/post/48512741
https://lemmy.ca/post/66154822
https://lemmy.ml/post/48503190
And yeah, that doesn’t even get into the posts that haven’t federated at all. I only went through a few pages of /new and I’m on posts that are hours old.
it’s not just hexbear, traffic here is noticeably & significantly down. i use lemmy at the same times every weekday and have developed a sense of normal based on comment frequency and it’s somewhere between 2x (for today) and 4x (for yesterday)
Nah, something’s wrong here on lemmy.ml. Compare these sites:
https://lemmy.world/?dataType=Post&listingType=All&sort=TopDay
https://lemmy.ml/?dataType=Post&listingType=All&sort=TopDayOn world it’s FuckAI post with just north of 900 upvotes, on ml it’s mostly local posts with the first one having 62 upvotes…
Same is true when comparing other instances, they all look similar to worlds TopDay sort, ml is the odd one out.
Dont know whats going on with these comments, but federation status shows up to date in both directions:
Yep. I checked that site and it definitely doesn’t show any issues, clearly everything is working.
Plenty of other folks are chiming in with the same experiences. It’s very clearly not just Hexbear. Is every other instance broken? Why are they federating just fine with each other, then?
https://lemmy.world/post/47958962
https://lemmy.ml/post/48512217
https://discuss.online/post/40968502
https://lemmy.ml/post/48508670
https://aussie.zone/post/33509821
https://lemmy.ml/post/48507829
https://piefed.blahaj.zone/c/womensstuff/p/792979/thoughts-on-this
https://lemmy.ml/post/48509886
https://startrek.website/post/40652133
https://lemmy.ml/post/48504964
https://lemmy.dbzer0.com/post/70286030
https://lemmy.ml/post/48512741
https://lemmy.ca/post/66154822
https://lemmy.ml/post/48503190
And yeah, that doesn’t even get into the posts that haven’t federated at all. I only went through a few pages of /new and I’m on posts that are hours old.
You’re right, its also really noticable if you just compare the All feed, on lemmy.ml almost everything from local users. Most posts from other instances are missing. In the server logs are a lot of errors like this:
lemmy-1 | 0: lemmy_server::root_span_builder::HTTP request lemmy-1 | with http.method=POST http.scheme="http" http.host=0.0.0.0:10633 http.target=/inbox otel.kind="server" request_id=780b6573-2672-4361-87c2-6ece46e272ee lemmy-1 | at src/root_span_builder.rs:16 lemmy-1 | 2026-06-09T17:34:25.537123Z WARN lemmy_server::root_span_builder: Unknown: Incoming activity has invalid signatureI thought at first that it would be related to the new 0.19.19 release, but downgrading to 0.19.18 doesnt fix the problem.
Issue link: https://github.com/LemmyNet/lemmy/issues/6581
Found the problem, I had commented out the
Hostheader in nginx thinking it would not be necessary. Turns out I was wrong. Adding the header back in fixed the problem.Big thanks for notifying about this!
OP already mentioned in the post body that the problem doesn’t seem to show up on that site. Just compare the TopDay/Active sort of different instances, lemmy.ml is missing a whole lot of full posts by now, not just comments.
I found it now, posts are coming in normally again.
