274
submitted 2 weeks ago* (last edited 3 days ago) by bobbiguy2122@lemmynsfw.com to c/lemmynsfw@lemmynsfw.com

Update 1: This is an issue with cloudflare, we are having issues with the account that hosts the site, we are contacting cloudflare support about the issue.


Update 2: we have switched s3 providers and images should be working again, older images may be missing, we will have to do a merge with the old s3 storage backup into the new provider, we plan to do this soon, but all new images will work as expected, sorry for the issues and thanks for your patience and kindness while we were working on the fix


Update 3: we have created a new ticket with cloudflare support with more information so that hopefully they take this issue more seriously.

For those experiencing issues uploading pictures, that is a separate issue we were not aware of until yesterday, that should be fixed now

EDIT: Federation issues:

It seems like federation is working now and the issues with photo uploading and admin settings getting 500 errors was also causing federation issues. Some posts were, to my understanding, in a federation window that has passed so the data is no longer there to be federated to our instance but future posts should federate properly. Please let me know if you see any other more recent posts that aren’t federating properly.

top 50 comments
sorted by: hot top controversial new old
[-] JadeBlossom@lemmynsfw.com 5 points 4 days ago

Well I hope we get our old images and stuff back cause I worked hard on all those and not wanting to reupload

[-] bobbiguy2122@lemmynsfw.com 2 points 3 days ago

I hope so too, however if you did end up wanting to re-upload them, you should be able to edit your posts and re-upload the photos if you still have the originals

[-] cavveman@lemmynsfw.com 3 points 4 days ago

Is update 3 part of why comments and up votes from other instances not showing? Attached screenshot is a comment I made in this topic but from Lemmy world account. It's not showing here yet.

[-] bobbiguy2122@lemmynsfw.com 2 points 3 days ago
[-] bobbiguy2122@lemmynsfw.com 2 points 4 days ago

No I'm not sure why that would be happening

[-] weird_alt@lemmynsfw.com 1 points 3 days ago* (last edited 3 days ago)

On my last post there were 3 comments from remote instances which were posted 2 days ago. They only appeared for me on this instance a few hours ago. I could see them on another account from a different instance before that.

Edit: also the upvotes jumped up simultaneously. Seems like everything was federated correctly from this instance, but had trouble federating back until today

[-] bobbiguy2122@lemmynsfw.com 2 points 3 days ago

It seems like federation is working now and the issues with photo uploading and admin settings getting 500 errors was also causing federation issues. Some posts were, to my understanding, in a federation window that has passed so the data is no longer there to be federated to our instance but future posts should federate properly. Please let me know if you see any other more recent posts that aren't federating properly.

We actually talked about this in the Lemmy matrix gen chat, if you'd like to join check the link for our matrix group in the site sidebar

[-] bobbiguy2122@lemmynsfw.com 1 points 4 days ago

Is this boost? Upvotes are working for me

[-] cavveman@lemmynsfw.com 1 points 4 days ago* (last edited 4 days ago)

Boost yes. But going through Web browser don't show it either.

[-] world_cavve@lemmy.world 1 points 4 days ago

Is update 3 part of why comments and up votes from other instances not showing?

[-] Clevererhans@lemmynsfw.com 3 points 1 week ago

Problems again?

[-] RedditRefugee69@lemmynsfw.com 45 points 2 weeks ago

I'm just happy to find out it wasn't an issue on my end.

It's been plaguing me for days and I was going to make a new account on a different instance to troubleshoot it today.

[-] NotAtAllSafeForWork@lemmynsfw.com 42 points 2 weeks ago

I thought it was my client being a shit. Thanks for all your work.

You are doing great and are appreciated.

[-] jessicablaze@lemmynsfw.com 30 points 2 weeks ago

Thank you for the update.

[-] lnsfw3@lemmynsfw.com 24 points 2 weeks ago

Godspeed fellow porn gremlins

[-] kokuen@lemmynsfw.com 22 points 2 weeks ago

Thanks for all your work!

[-] IAmNorRealTakeYourMeds@lemmy.world 18 points 2 weeks ago

thanks for your hard work

[-] IWW4@lemmy.zip 15 points 2 weeks ago

I figured it was my connection.

Keep doing what you do!

[-] leftzero@lemmynsfw.com 15 points 2 weeks ago* (last edited 2 weeks ago)

The problem seems to be that for some reason you're messing with image urls and adding https://lemmynsfw.com/api/v3/image_proxy in front of them for some nefarious reason, which obviously prevents them from working.

Just stop messing with people's posts and everything will work fine.

There's clearly something both extremely ill intentioned and extremely incompetent happening here.

This is precisely the kind of profoundly stupid meddling I left reddit for.

Time to look for a new instance or quit lemmy altogether, I guess.

It was... somewhat almost okayish while it lasted.

[-] MisterSinn@lemmynsfw.com 47 points 2 weeks ago

I'm just going to include a section from the patch notes for Lemmy version 0.19.4.

There is a new config option called image_mode which provides a way to proxy external image links through the local instance. This prevents deanonymization attacks where an attacker uploads an image to his own server, embeds it in a Lemmy post and watches the IPs which load the image. Instead if image_mode is set to ProxyAllImages, image urls are rewritten to be proxied through /api/v3/image_proxy. This can also improve performance and avoid overloading other websites...

For the issue at hand, it appears to be related to Cloudflare based on the 403 error message when tying to view an image.

[-] leftzero@lemmynsfw.com 9 points 2 weeks ago* (last edited 2 weeks ago)

Sure, the visible symptom of the issue is cloudflare blocking the image proxy, probably with good reason, since the explanation is absurd (except for the deanonymization part, which is just schizophrenically paranoid; no one cares enough about who's looking at some random image to waste their time setting that up), but the unnecessary and nonconsensual meddling with the urls is the root cause.

We're talking Lemmy here. As great as federation is, small self hosted instance servers will always provide less performance and get overloaded faster than whatever CDN the site the user is linking to is using, so that argument is evidently fallacious.

(Plus, the option to host the image on the instance server has always been there: just download the fucking image from wherever you found it and upload it to the fucking instance. If anything, what this does is take away that choice from the user, leaving us with the “choice” to upload the image... or have it silently uploaded for us anyway.)

Let's be serious, the only reasonable motive behind this (especially when you take into account the devs' notorious ideology) is to be able to better control what the users post.

The deanonymization bit falls by its own weight, since, sure, the original hoster can't see who's loading the image (not that they ever cared to to start with), but now the instance admins (and / or the devs) can. Nothing is ever anonymous in the cloud, for fucks sake. Again, this is just taking away the choice of who to trust, and making lemmy look like the most untrustworthy option in the process.

The most important part, though, is that by highjacking the image hosting without the user's knowledge (and against the user's will, since, again, we could always choose to host the image on the instance, and this applies specifically to the case where the user did not intend to host it here), the instance (and / or the devs) gets control over what image gets actually served.

Enshittification happens. Every single image in the cloud will, sooner or later, be replaced with an ad. That's as certain as the third law of thermodynamics. When you link to a cloud hosted image, you're (mis)placing your trust on the hoster to keep serving that version of the image for the foreseeable future. Maybe I trust the lemmy instance more than the original site, in which case I'll upload the image. Maybe I trust the site more, in which case I'll link it. Maybe I trust neither, and I'll self-host the image, and link it (which is almost certainly the best option for people posting images of themselves, as is the main intended case for lemmynsfw).

But those two later options are now gone. Stolen from us, the users. And, obviously, I (and hopefully most other users) no longer trust the instance, or lemmy. Now the instance (and / or the devs) always has the option to change the image, instead of only when we misplaced our trust on them.

Plus, as the current kerfuffle so evidently shows, it adds a completely unnecessary extra point of failure.

The images would work perfectly if they weren't being shoveled through a hostile proxy no one asked for which is being blocked by cloudflare, probably with good reason.

The lemmynsfw admins could trivially solve the issue for newly linked images by disabling this stupid malicious option (already uploaded ones would probably require fixing the mangled urls at the database level, which is the least that they deserve for having enabled it in the first place), but they're not, they're trying to get cloudflare to fix it, a well known sisyphean task, i.e., an evident waste of everyone's time.

But they're not, so they clearly want to keep the proxy, the very root of the problem.

The whole thing is therefore not only malicious, but profoundly stupid, and depressing.

Just like good old reddit. 🤢

[-] bobbiguy2122@lemmynsfw.com 38 points 2 weeks ago

That is not the issue, we are having issues with our cloudflare account and our S3 bucket, we are in contact with support and working on a fix. You are making wild assumptions.

[-] monkeyman69@lemmynsfw.com 21 points 2 weeks ago

It must be hard to deal with these kind of paranoid nutcases, thanks for your hard work.

[-] leftzero@lemmynsfw.com 12 points 2 weeks ago* (last edited 2 weeks ago)

What fucking hard work?

They're just waiting for a response from cloudflare that almost certainly will never come, when they could fix the issue by simply removing the proxy.

You can check it for yourself, look at the URL of any random external image they're serving, https://lemmynsfw.com/api/v3/image_proxy?url=https%3A%2F%2Flemmy.world%2Fpictrs%2Fimage%2F301480eb-7622-41f3-a66a-a868c3544b90.jpeg, for instance.

Opening that will give you an error instead of an image, because cloudflare is being cloudflare and causing the proxy to fail.

Remove the unnecessary proxy part and fix the mangling, however, and it'll work perfectly: https://lemmy.world/pictrs/image/301480eb-7622-41f3-a66a-a868c3544b90.jpeg.

(The link is just some random image from the front page, irrelevant but harmless.)

This could and should have been fixed days ago, and would never have happened if they hadn't tried to hijack the urls in the first place!

(Well, it might still have happened with local images, but at least external ones would still work, instead of the whole instance being unusable.)

I think the part that makes your posts seem hostile is that you either know and haven't stated, or are assuming, that this is an intentional behavior on the part of lemmynsfw.

Was this behavior added to Lemmy's master codebase, or is there evidence lemmynsfw's admins customized the code to add a Cloudflare proxy for images? Either way, it doesn't seem like lemmynsfw is acting in bad faith.

I get that there's a larger Reddit / enshittification narrative you're viewing this through, but if you want them to not do that thing - use the URL-cloud-ifier - you'd be much more persuasive and probably get a more constructive reply if you just explained why it's a bad thing and asked them not to do it.

load more comments (1 replies)
[-] happyHarryHardon@lemmynsfw.com 13 points 2 weeks ago

Some people demand a lot. I think most of us are happy just knowing whats going on.

Any way thanks!

load more comments (8 replies)
[-] leftzero@lemmynsfw.com 7 points 2 weeks ago* (last edited 2 weeks ago)

That is not the issue

It's the underlying cause of the issue. (Well, that and dealing with cloudflare.)

we are having issues with our cloudflare account

Yes (that's what cloudflare appears to be for). Which is causing your stupid proxy to produce this shit instead of an image:

{"code":"object-request-error","msg":"Error performing PUT https://eb21b514652257169b552d5ece7ddfd2.r2.cloudflarestorage.com/lemmynsfw/01/98/22/63/d5/9e/74/c2/94/48/97232eff1bb0.jpeg in 45.917481ms - Server returned non-2xx status code: 403 Forbidden: NotEntitledPlease enable R2 through the Cloudflare Dashboard."}

You'll notice how the first link, which is what your site produces, craps out (because your cloudflare account isn't working properly, as tends to happen with cloudflare, no surprise there), while the second link, which is the original URL reconstructed from your mangling, works perfectly. (I grabbed it from some random post in the front page, the content is irrelevant but harmless).

So keep the fucking original URL instead of hijacking it with some stupid unnecessary proxy and it'll fucking work!

Q.E. fucking D., for fuck's sake! 🤷‍♂️

[-] himitsu@lemmynsfw.com 10 points 2 weeks ago

You’re an idiot. Proxies protect privacy. That’s literally why you’re here and not on Reddit, right?

[-] leftzero@lemmynsfw.com 8 points 2 weeks ago

Calling it a proxy doesn't mean it is one.

We have no way of knowing what's behind https://lemmynsfw.com/api/v3/image_proxy. All we have is the admins' word that it is one (wait, no, not even that, since they've never told us about it to start with).

All we know is that our urls are being hijacked (and that this is causing them not to work, and that the reason the devs — which are known to be untrustworthy — tried to justify this option with doesn't hold water) without anyone asking us first or warning us..

If I care about privacy I'll take care of that. I'll upload my images somewhere I trust or, better yet, control. If I want a proxy or CDN it'll be on my terms.

Even supposing it is a proxy, does it have a cache? Or a CDN? The whole cloudflare kerfuffle seems to suggest it does. How often do they update? If I decide to remove my image from wherever I hosted it, how long will it take for the cache to reflect that?

I don't know; lemmynsfw never told me, just like they didn't tell me they were using this alleged proxy and hijacking my urls.

Even if we apply Hanlon's razor and assume the lemmynsfw admins didn't know they had this option turned on and the devs just snuck it in by default in an update, the fact that they've kept it turned on for days when turning it off would fix the issue for external images makes the ignorance excuse moot.

They know it's on, they want it on, and they've got no intention of telling anyone or asking if we're fine with it.

This destroys any trust we could have had on them, and makes moot any assurance on their part (if they ever made one, which they have not) that this alleged proxy is benign.

load more comments (2 replies)
load more comments (1 replies)
load more comments (5 replies)
load more comments (1 replies)
[-] spinne@lemmynsfw.com 9 points 2 weeks ago

Nefarious or not, I appreciate you calling out the problem

load more comments (1 replies)
[-] sasoni75@lemmynsfw.com 10 points 1 week ago

Just to inform the admin, uploading photos does not work, I get an error(I copied it if you want it), and community photos in the side won't show up. Thanks for the hard job!

[-] BigMcLargeHuge@lemmynsfw.com 10 points 2 weeks ago

It's always DNS.

[-] ProteanG6777@lemmynsfw.com 9 points 1 week ago

well done, thanks to your team.

[-] ExhibiCat@lemmynsfw.com 9 points 2 weeks ago

I'm surprised some images still work though, even recently posted ones.

Thank you for all the work on this and now importantly on building and maintaining lemmynsfw in the first place.

[-] Sagan_Wept@lemmynsfw.com 8 points 2 weeks ago

Waves fist at the clouds ✊☁️ Damn you cloudflare!!!

[-] leftzero@lemmynsfw.com 7 points 2 weeks ago

cloudflare

I don't understand why this malware is allowed to exist.

Their only purpose seems to be to stop the Internet from working properly.

load more comments (3 replies)
[-] Big_Boss_77@lemmynsfw.com 8 points 2 weeks ago
[-] bobbiguy2122@lemmynsfw.com 16 points 2 weeks ago
[-] Big_Boss_77@lemmynsfw.com 7 points 2 weeks ago

Awesome! Thank you so much!

[-] cherry001@lemmynsfw.com 7 points 1 week ago* (last edited 1 week ago)

Great job👌👌 Thank you for your efforts

[-] Avelo@lemmynsfw.com 7 points 2 weeks ago

Thank you so much.. Have a nice weekend

load more comments
view more: next ›
this post was submitted on 16 Jul 2025
274 points (100.0% liked)

Lemmy NSFW

11711 readers
3 users here now

Updates about lemmynsfw.com

founded 2 years ago
MODERATORS