this post was submitted on 05 Jan 2025
56 points (92.4% liked)

Fediverse

28835 readers
354 users here now

A community to talk about the Fediverse and all it's related services using ActivityPub (Mastodon, Lemmy, KBin, etc).

If you wanted to get help with moderating your own community then head over to [email protected]!

Rules

Learn more at these websites: Join The Fediverse Wiki, Fediverse.info, Wikipedia Page, The Federation Info (Stats), FediDB (Stats), Sub Rehab (Reddit Migration), Search Lemmy

founded 2 years ago
MODERATORS
 
top 20 comments
sorted by: hot top controversial new old
[–] Shadow 69 points 2 days ago* (last edited 1 day ago) (3 children)

Our hardware failed this morning and we're stuck waiting on OVH to investigate the support ticket I filed. They claim their support is 24/7 but live chat / phone is just business hours, so worst case we might be down until tomorrow =(

https://status.lemmy.ca/ has an update, but not much to say really except "It's down until it's not"

Edit: I'm assuming we're down until I can call / chat with them first thing in the morning, so expect to be back up early tomorrow (depending on how long the parts replacement takes).

Edit 2: I called them when I woke up at 6:30 PST and hopefully they'll fix it soon now....

[–] [email protected] 15 points 1 day ago (1 children)

They claim their support is 24/7 but live chat / phone is just business hours

It's increasingly common for "24x7 support" to mean access to their KB/FAQ and the ability to raise a ticket, while the actual support is a skeleton crew working regular hours.

Not unlike "your call is very important to us..." πŸ˜…

[–] [email protected] 8 points 1 day ago

You can send us email at any time of day or night!

[–] Bakkoda 31 points 2 days ago

Just changed all my maintenance messages to

It's down until it's not

[–] JohnnyCanuck 7 points 2 days ago

Thanks! I'll bookmark that.

[–] [email protected] 16 points 2 days ago (2 children)

Weird to think what happens when you lose a site you visit on a regular basis. I can understand the whole instance going down but us users just get lost when things get turned off. There's no way to know what happened, why or when or if anything will return.

I had to resurrect my old lemmy.world account to figure out what happened. I guess we have to hang on to two or more unique instance accounts in case our main instance goes down.

Stay well guys and I totally appreciate the admins of lemmy.ca and all the work they do. I'll patiently wait and look forward to seeing it online again.

[–] [email protected] 7 points 1 day ago

At least it’s not like Reddit when the website goes down you can’t use it at all.

[–] Shadow 9 points 2 days ago (2 children)

I keep trying to open the lemmy app on my phone out of habit.

[–] JohnnyCanuck 3 points 1 day ago

I did that five times before remembering I had an old account on this instance that I could switch to to post the question and see what communities I used to be subscribed to.

[–] [email protected] 6 points 2 days ago

It feels shitty and I was doing the same thing all day. I had to resurrect my lemmy.world account just to get back online. It's made me realize that I should maintain both accounts for shut downs like this. And now I'm also having to do more in maintaining my Mastodon account too.

[–] [email protected] 16 points 2 days ago (1 children)

They posted on the admin chat that their hosting provider had a hardware failure and is working on providing a replacement server.

[–] [email protected] 10 points 2 days ago

It may not get fixed untill tommrow since they are not getting any response from their service provider.

[–] [email protected] 15 points 2 days ago* (last edited 2 days ago) (1 children)

That's an error message from Cloudflare, their tunnel provider. The server is probably unreachable. Either they're doing some maintenance, or it's a hiccup or something went wrong. You need to wait for a moment. Or maybe a bit longer if they're facing any issues.

[–] [email protected] 3 points 1 day ago* (last edited 1 day ago) (1 children)

Wow, they're really offline for a long time now. Did OVH really mess up that hard? I guess hardware in a datacenter fails every day? Wouldn't they be prepared for that and just restore/migrate to a different machine, then?

[–] [email protected] 2 points 1 day ago (1 children)

Yeah that's usually how it's supposed to work, but sometimes it doesn't work properly for a variety of reasons.

[–] [email protected] 3 points 1 day ago* (last edited 1 day ago) (1 children)

Yeah, it's speculation at this point anyways. I mean I've only ever seen webservers with a RAID. So one harddrive failure shouldn't affect it too much. Then most people rent a VPS and that usually runs on some cluster. Even if a whole machine goes down, it's supposed to come up again on some other machine automatically within a few minutes. And then you should have tech support of a big hosting provider answer within some timespan. I hope someone will write a summary of the events and link it somewhere. Maybe a lot of things went wrong. Or it's some kind of error in the specific setup lemmy.ca did.

[–] Shadow 3 points 1 day ago

It wasn't a hard drive failure but something more severe, since the remote console is non-responsive. I just called OVH to yell at them and press on the ticket, so hopefully should be resolved soon.

[–] [email protected] 6 points 2 days ago (2 children)

You can host a lemmy instance through a cloudflare tunnel??? :0

[–] [email protected] 12 points 2 days ago

Yes of course, as almost any other website.

[–] [email protected] 2 points 1 day ago

This very message is coming to you through a tunnel!