this post was submitted on 12 Jun 2023
617 points (99.8% liked)
Technology
1928 readers
7 users here now
Rumors, happenings, and innovations in the technology sphere. If it's technological news, it probably belongs here.
Subcommunities on Beehaw:
This community's icon was made by Aaron Schneider, under the CC-BY-NC-SA 4.0 license.
founded 2 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
I'd love to know what it is about subreddits going private that caused issues.
Maybe some overload caused by a process having to dig deeper to find best/top posts?
apparently that's exactly the case.
That is an interesting aspect no engineer could have foreseen!
You'd be surprised how much critical infrastructure was implemented through trial and error and has just been left like that for years...
Anything less than 99% of infrastructure working that way would be surprising. Everything is held together with scotch tape and scotch whisky.
I'll be sure to repeat that last line to my fellow team members :D
I like this idea. I imagine that with the top subs being dark the automated top posts that get scrounged up may be too terrifying for the front page and they hit the panic button while they scramble to curate through the absolute worst filth they've ever seen.
“It’s merely coincidence. But starting Wednesday, our servers will be more robust and you can browse the site using our official app.” - Spez, while sniffing a decanter of human shit
God we need indefinite blackouts.
It’s entirely possible that they’ve made some assumptions about what a “normal” level of traffic looks like when writing code for their backend, which has caused some things to break when that has changed.
Not our fault if their code is shit.
How is that an example of bad code?
Honestly, it’s probably not - if I’m actually right this is likely an issue that Reddit’s engineers never predicted would happen so never planned for it. I was being hyperbolic.
It's not reactive. A proper reactive system can handle fluctuations in usage patterns more robustly.
I'm having a hard time believing the claim that Reddit's code isn't reactive.
Wouldn't be surprised if it's just a gigantic mess of nested if-else statements.
Gotos all the way down
Maybe, but this was a huge increase in usage. Reddit never expected to deal with anywhere near thousands of subs going private simultaneously.
The servers run on the tears of bitter whiny CEOs.
Reddit is hosted on AWS after all...
Tildes' dev Deimos used to work for Reddit and had this guess https://tildes.net/~tech/163e/reddit_appears_to_be_down_during_blackout_day_1#comment-87v1.
They're lying. Fish swim, birds fly, sun shines, Reddit lies.
Probably a drop in usage flagged some internal test