109
submitted 9 months ago by cole@lemdro.id to c/android@lemdro.id

Google today announced a handful of wearable and navigation updates, starting with public transit directions in Google Maps for Wear OS.

71
submitted 9 months ago by cole@lemdro.id to c/android@lemdro.id
95
submitted 9 months ago by cole@lemdro.id to c/android@lemdro.id

We typically like Pixel phones a lot, but we have some reservations about Google's quality control

61
submitted 9 months ago by cole@lemdro.id to c/android@lemdro.id

Google Maps is changing with pretty significant redesigns across key surfaces, including when searching for directions...

[-] cole@lemdro.id 16 points 9 months ago

I don't understand why this exists. Polestar has enough problems lol

[-] cole@lemdro.id 15 points 11 months ago

and to note hasn't seen any progress or work in 3 years

[-] cole@lemdro.id 40 points 11 months ago* (last edited 11 months ago)

Basically, the lemmy backend service for some reason marked every instance we federated with as inactive, which caused it to stop outbound federation with basically everyone. I have a few working theories on why, but not fully sure yet.

TL;DR lemmy bug, required manual database intervention to fix

This was a stressful start to a vacation!

For a more detailed working theory...

I've been doing a lot of infrastructure upgrades lately. Lemdro.id runs on a ton of containerized services that scale horizontally for each part of the stack globally and according to load. It's pretty cool. But my theory is that since the backend schedules inactive checking for 24 hours from when it starts that it simply was being restarted (upgraded) before it had a chance to check activity until it was too late.

theory:

  • scheduled task checks instances every 24 hours

  • I updated (restarted it) more than every 24 hours

  • it never had a chance to run the check

  • ???

  • failure

This isn't really a great design for inactivity checking and I'll be submitting a pull request to fix it.

1
Recent Downtime (lemdro.id)
submitted 11 months ago* (last edited 11 months ago) by cole@lemdro.id to c/lemdroid@lemdro.id

There was a brief (~5 minute) period of downtime on lemdro.id recently. This occurred during a routine database upgrade when some strange issue caused a member of the cluster to become inconsistent and refuse to form quorum.

This locked the entire cluster into an invalid state which took some troubleshooting to fix. My apologies.

I will be rolling out read replicas for folks on the East coast of the US as well as those in Europe sometime in the next week, you should notice a pretty dramatic reduction in latency if you are from those areas. Additionally, other recent changes have increased reliability and decreased latency which may or may not be noticeable.

I will post another update before I start rolling out the read replicas since it is kind of a big change (and I will schedule a time for it)

[-] cole@lemdro.id 15 points 11 months ago

not surprising, but super disappointing. Beeper Mini was a dream come true

1
submitted 11 months ago by cole@lemdro.id to c/lemdroid@lemdro.id

I recently rolled out an infrastructure upgrade this last Monday (Dec 4) with the intent of reducing peak response times and removing occasional scaling errors.

Unfortunately, my metrics system showed slightly elevated error rates, so I've decided to rollback these changes for now. I will make another announcement before I roll them back out in the future. Thanks for your support!

[-] cole@lemdro.id 16 points 1 year ago

eBay doesn't have workers, it's a marketplace that facilitates transactions between sellers and buyers

[-] cole@lemdro.id 18 points 1 year ago

I've gotten an email like this before for lemdro.id. I think it's a generic phishing email since the community links look like email addresses (and actually often are)

1
submitted 1 year ago* (last edited 1 year ago) by cole@lemdro.id to c/lemdroid@lemdro.id

Hey there everyone. I think the Photon project has matured enough to the point where I feel ready to replace the default lemmy frontend with it. Since this instance serves roughly 1000 people now, I figured this was worth holding a vote on!

Please check out Photon as currently hosted at https://p.lemdro.id.

If you support changing the default frontend to Photon, upvote my comment on this post. If you don't support it, downvote that same comment.

Thanks!

[-] cole@lemdro.id 19 points 1 year ago

Freesync yes, HDR soon™

9
submitted 1 year ago* (last edited 1 year ago) by cole@lemdro.id to c/androidmemes@lemdro.id

RIP :(

4
submitted 1 year ago by cole@lemdro.id to c/aistuff@lemdro.id
[-] cole@lemdro.id 18 points 1 year ago

It is my opinion that there is only "tension" because you say there is. I do not believe that most people feel that way; it isn't exactly an active war zone. Nevertheless, I respect your decision as moderator of this community, and I'll leave you be in the future. Best of luck!

[-] cole@lemdro.id 16 points 1 year ago

I'm the backend guy at lemdro.id, I started the instance. I don't moderate the !android@lemdro.id community (or any of them... I purely do infrastructure), but I am quite aware of the drama around recent events involving the two android communities.

The intent behind banning all lemdro.id users from applying to be mods feels quite... retaliatory? I'm not sure there is anyone on lemdro.id who would apply, but depriving them the opportunity simply because they are associated with an instance that hosts a community which had another community merge into it?

It is your community, you can of course do what you want. I just wish we could drop the drama, act like adults, and move on. Tired of the vitriol.

289
Android rule (lemmy.blahaj.zone)
submitted 1 year ago by cole@lemdro.id to c/androidmemes@lemdro.id
1
submitted 1 year ago* (last edited 1 year ago) by cole@lemdro.id to c/lemdroid@lemdro.id

Earlier today, I identified the root cause of an issue causing annoying 502 errors intermittently. If you've ever had an action infinitely load until you refreshed the page, that is this issue. I deployed a fix, and am slowly scaling it down to stress test it. If you encounter an infinite loading occurrence or an HTTP 502 error please let me know!

UPDATE: Stress testing complete. Theoretically we should be equipped to handle another 5k users without any intervention from me

1
submitted 1 year ago* (last edited 1 year ago) by cole@lemdro.id to c/lemdroid@lemdro.id

Hello folks! I am migrating the image backend to an S3-compatible provider for cost and reliability reasons. During this time, thumbnails and other images hosted here will be borked, but the rest of Lemdro.id will remain online. Thank you for your patience!

UPDATE: Image migration is gonna take a hot minute. Should be done in around 6 hours, I'll get it fully fixed up in around 7-8 hours when I wake up (~08:30 PDT)

UPDATE 2: It failed, yay! Alright, fine. I turned the image proxying back on. I am migrating to S3 in the background and will switch over when it is done. Any images uploaded in the next 8 hours or so may end up being lost.

UPDATE 3: Migration complete. Will be rolling out the update to S3-backed image storage in around 6 hours (~6pm PDT)

UPDATE 4: Object storage backend deployed! Thanks for your patience folks.

[-] cole@lemdro.id 15 points 1 year ago

test comment please ignore

[-] cole@lemdro.id 20 points 1 year ago

sure! take a look at this issue: https://github.com/LemmyNet/lemmy-ansible/issues/106

It is likely yours is fixed if you deployed recently

[-] cole@lemdro.id 88 points 1 year ago

Hello! Admin here at lemdro.id. This is the result of several problems in the default reference nginx config. I am working on resolving this right now and should have it fixed within the next 30 minutes!

view more: next ›

cole

joined 1 year ago
MODERATOR OF