this post was submitted on 12 Jun 2023
163 points (100.0% liked)
Technology
1928 readers
7 users here now
Rumors, happenings, and innovations in the technology sphere. If it's technological news, it probably belongs here.
Subcommunities on Beehaw:
This community's icon was made by Aaron Schneider, under the CC-BY-NC-SA 4.0 license.
founded 2 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
I find it interesting that so many subs going private caused issues with the entire site. I'm not a programmer/tech person, but I don't really understand why that would happen. Did it just create a bandwidth bottle neck or something as users flooded non-blacked out subs?
I'm a backend server programmer, and although I have never worked on resdit I can imagine the conversation a few years back:
Dev A: How do we implement private? Should each post have a private flag or do we join on the subreddit to get it?
Dev B: that join might be heavy for the front page since we might have several different subreddits, a flag in each post is faster.
Dev A: But that means that whenever a subreddit goes private we need to flag all of the posts there
Dev B: yeah, but that doesn't happen often, the front page gets accessed millions of times, most subreddits never change status, especially larger ones.
Dev A: I guess you're right, even a large subreddit will be done in minutes, what are the odds of several large subreddits going private at the same time?
The funniest thing about this is that means that the subreddits in question could keep flipping the switch back and forth and with each change cause outages.
back and forth, forever...
That's r/talesfromtechsupport material lol.
We don’t know what switching a sub to private, will trigger on their backend. Maybe there is some internal shifting of data somewhere, or something similar to this. Doing this, on a lot of the big subs, could lead to large-scale performance issues.
Interesting/fascinating. I have no idea how development works, but it's interesting to me to see the impact. Someone else mentioned that it could be resource allocation locking up too, which makes sense when I think about it more. A subreddit with subscribers in the millions would need more resources allocated to it due to the increased demand, so if they're suddenly unavailable I can kind of understand why/how that would creat issues. All very neat.
I'm a backend developer and have to deal with this nonsense at my job. There are lots of options, each with tradeoffs, such as:
And if you go for one of the two last options, you can reduce the hit when switching by being eventually consistent, meaning you don't change everything all at once but instead set the flag or change the URL in batches over a period of time. That takes effort to get right, and there's going to be inconsistency until everything is done.
So there are multiple ways to do it, and each way has a big chance at devs messing something up when they don't consider something like a bunch of subreddits going private all at once.
Or it could just be a run of the mill DDOS because some redditors wanted to troll Reddit.
Nah I think they just knew there'd be a drop in traffic and scheduled some maintenance, lol
If you read the article there's a quote that distinctly mentions unexpected issues not planned maintenance.
Yes. Let us trust what Reddit staff say about it. That feels logical.
I mean in this case, you can decide between some theory based on no information at all but unaffiliated, or based on all information but biased.
There's still a good chance that the downtime is unexpected, downtime in general is never good for a site that earns money by people being on it.
So I think the "unexpected" case is still more likely than it being intentional.
Maybe they optimised the site to have certain servers allocated to the bigger subs, and with them all going dark it screwed with that balance or something?
I would guess it's the influx of traffic from all the news coverage, combined with people repeatedly refreshing thinking the site is "broken."
I'd imagine that would be offset by all the people who are redditing much less because most of the subs they frequent are private?
I would imagine this also, honestly.
Maybe an issue on the back end skipping over all of the new private subs? I assume that many of the big subs had popular posts just before they went private, so the server has to spend more time skipping over private posts when rendering popular?
The theory is that the way Reddit caches its subreddit content for the front page didn't jive with a bajillion subs being gone all at once when America woke up this morning.