this post was submitted on 28 Jun 2023
432 points (98.2% liked)

Lemmy.World Announcements

28961 readers
4 users here now

This Community is intended for posts about the Lemmy.world server by the admins.

Follow us for server news ๐Ÿ˜

Outages ๐Ÿ”ฅ

https://status.lemmy.world

For support with issues at Lemmy.world, go to the Lemmy.world Support community.

Support e-mail

Any support requests are best sent to [email protected] e-mail.

Report contact

Donations ๐Ÿ’—

If you would like to make a donation to support the cost of running this platform, please do so at the following donation URLs.

If you can, please use / switch to Ko-Fi, it has the lowest fees for us

Ko-Fi (Donate)

Bunq (Donate)

Open Collective backers and sponsors

Patreon

Join the team

founded 1 year ago
MODERATORS
 

We've upgraded lemmy.world to 0.18.1-rc.1 and rolled back that upgrade because of issues.

(If you had posted anything in those 10 minutes between upgrade and rollback, that post is gone. Sorry!)

The main issue we saw is that users can't login anymore. Existing sessions still worked, but new logins failed (from macos, ios and android. From linux and windows it worked)

Also new account creation didn't work.

I'll create an issue for the devs and retry once it's fixed.

Edit Contacted the devs, they tell me to try again with lemmy-ui at version 0.18.0. Will try again, brace for some downtime!

Edit 2 So we upgraded again, and it seemed to work nicely! But then it slowed down so much it was unuseable. There were many locks in the database. People reported many JSON errors. Sorry, we won't be on 0.18.1 any time soon I'm afraid..

you are viewing a single comment's thread
view the rest of the comments
[โ€“] [email protected] 8 points 1 year ago (1 children)

appreciate the transparency. how are things looking in the back in lemmy.world (server wise)? will we get to a point where it wont require complete rollbacks on the state when a botched update gets rolled out?

[โ€“] [email protected] 1 points 1 year ago

Not having to do a database rollback is a really, really hard problem to solve, and it would almost certainly need to be on the Lemmy developers side, not the server owner's side. And if I'm them, that's a low priority issue, and probably not something I even think about until 1.0.

Basically, they write code that says what to do in the event of a database version change. Usually this only handles upgrade cases, because that's what happening most of the time. One example of something you might do in a db upgrade is let's say you had a column where the data type was only numbers, but now you want to allow any alphanumeric character for some reason. You could have a line of code that converts the number to a string.

Okay, but now you need to go back to the previous version. Okay, your db change code runs, but it's the old version of the db change code, not some new version that you wrote. You unfortunately didn't have a crystal ball when you wrote this code and couldn't predict that you were going to change the data to strings, so you didnt write code to change it from a string to a number.

This is why most software doesn't support downgrades unless you wipe first. For example, if you updated your aging MacBook to the latest Mac OS version, then realized it slows down your laptop too much, you can only go back if you first wipe your laptop in the process. So it's just easier to just take a snapshot before an upgrade and revert to the snapshot if it fails. Some folks will even do "scheduled maintenance" time during the upgrade in which the whole system goes down for a short time so they don't have to risk losing data that happened after the snapshot.