427
Lemmy.world down, probably following the upgrade. A reminder to move to smaller instances for a better experience
(files.catbox.moe)
this post was submitted on 29 Jul 2023
427 points (92.8% liked)
Fediverse
17795 readers
57 users here now
A community dedicated to fediverse news and discussion.
Fediverse is a portmanteau of "federation" and "universe".
Getting started on Fediverse;
- What is the fediverse?
- Fediverse Platforms
- How to run your own community
founded 5 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
A reminder that this constant advice people blindly parrot to install and flock to smaller instance has now created something like 1000 new servers in 50 days that are poorly run and already going offline as quickly as they went online.
Github Issue 2910 is the kind of PostgreSQL problems that the developers ignored for months and people still defend the developer choices to have the code doing real-time counting of every single comment and post for numbers nobody needs to needs done in real-time.
PostgreSQL is voodoo to this project, they do everything they can to avoid going to [email protected] community and asking for help, learning 101 about how to fix their SQL TRIGGER logic like Github Issue 2910 spelled out June 4.
I know you are salty about how you are getting treated over at GitHub, but you should look at it objectively, Blaze is clearly advocating that people join top instances that's not lemmy.world or lemmy.ml, not nobody instances that only have 1-2 users. They certainly aren't going offline as quickly as they come online.
No, it isn't about my personal treatment. It's about the cultist attitude you have towards Lemmy and the leaders without any ability to see what they are doing behind the scenes with the code. I know cults and religious faith is how many people enjoy the world.
A 2-line SQL TRIGGER removal takes about minutes to fix. It was crashing the entire site constantly. They sat by and asked for donations of money.
Then go fix it and open a PR
Do you think I am the one who created the mistake or something? That I have access to the servers to install it?
It's so odd to me that you respond this way, as if it was my coding mistake. It isn't even me who opened issue, that is GitHub "makotech222" - is that your answer to them?
I don’t believe they’re insinuating that you were the one that created the mistake. Rather, that you seem to be knowledgeable of the specific problem and may be the one most capable of fixing it. The two line fix may be obvious to you, but may not to the main Lemmy devs. Until phriskey got involved, a lot of db tuning was being avoided (they’re responsible for most of the big db improvements this version).
and I did not understand or properly relate to that project culture. It had been that way for years and I should have "read the room" "go with the flow".
No, it's everything to do about your personal treatment, stop deceiving yourself. Just because you claim you have autism doesn't immediately grant you the right to be entitled. You don't get your way so you spam create multiple issues to call out the developers, and you expect people to believe it isn't personal for you?
If you aren't happy with the Lemmy developers, fork the project, run your own fork, convince others to use your fork. It's a FOSS world, no one has to do what you say, even if you claim to be autistic.
Telling someone "you claim you have autism" is extremely ableist to all disabled people.
What is a SQL trigger and why is it taking down servers? Do you know how to fix it?
I think 0.18.3 fixed some of it, but there are likely some more performance issues related to PostgreSQL lurking in Lemmy.
A TRIGGER in SQL is a logic that executes based on other activity.
Lemmy uses them so that when you create a new comment or post, it executes code to insert tracking record for votes and comments on a post. One of the things Lemmy does is called site_aggregates, and there was a bug where it was updating the counts for 1500 servers instead of just the one server. That got fixed in 0.18.3
Deleting accounts in lemmy was causes crashes. I'm not sure if that has been entirely resolved. These things are all kind of hidden in the background of the code, so a lot of developers overlooked that there were problems in them.
Isn’t there a logstream they could tap into to have a separate async tally going on instead of doing it synchronously? Probably a lot of things could be delegated to an async job performed when server load allows?
I've had to really adjust my thinking with this project. They want to do things a very particular way and it goes back 4 years, and a lot of the mistakes are just now getting noticed/attention. For example, comments were not deleting on all the servers, I was testing that after comparing server copies of the same communities and found they were not the same. It just didn't seem to have a lot of people spot-checking it for mistakes. I am learn to just "go with the flow" and face that it's more like how musicians would approach design and running a project. Media-focused systems can be that way.
Going with the flow is both ways. It’s a community project so everyone’s opinion matters, especially those who contribute PRs.
I am always advocating for any of the top 25 instances that are not Lemmy.world or Lemmy.ml
For the rest of your post, I don't know what that has to do with people aggreating on LW.
aggregation refers to the lemmy database tables, site_ aggregates, community, person. The SQL TRIGGER logic lemmy_server uses that has been the source of so many crashes the past 60+ days.
Even if the SQL was top notch, it would not be a good thing for 50% of the active users to be on a simple instance. Just makes it easier to take down by any potential attacker.
Shit dude, you still going on about the GitHub issues?
You are really fucking stupid and it shows.
And this will always.. always be the biggest problem in the FOSS community.
"I dont like X, so I'm going to leave and make my own version of X"
So userbases get spread thin, manpower gets spread thin, developers get spread thin, and the user experiences degrades for everyone until it pushes them back to the bullshit websites and products.
This is exactly what federation is meant to solve: power in numbers without the centralization. Is that so hard to understand?
Sometimes I question why people not in favor of the decentralization are commenting on a Fediverse platform. Why not go to Tildes, Squabbles or another centralized alternative? There is plenty of fish in the sea.
This is another big problem in the FOSS
"If you dare offer valid criticism, then why are you even here? get out and go somewhere else!"
Your criticism is nonsensical. It's literally criticizing the purpose of the project.
Oh dear, are you gonna start another tissue shortage with all these tears?
Pot calling the kettle black. I offer criticism of your criticism and you throw a hissy fit. Poor child.
Wow, I havent gotten a genuine, unironic "No, U!" since like kindergarten.
Wooosh
Your answer didn't justify lemmy.world being treated the same as Lemmy as a whole. It's just a bunch of people who don't understand federation.
And, factually, the project leaders telling everyone to create 1000 new instances and shutting down sign-up on Lemmy.ml caused more performance problems.
They had a bug in their PostgreSQL TRIGGER logic where 1500 instances were updating + 1 comment and +1 post counting instead of WHERE site_id = 1, a single database row. So each new Lemmy server that went online made the table larger and crashes more frequent on lemmy.ml
The amount of disk writing by lemmy was ignored
They've neither told people to create 1000 new instances, nor have they closed signups on lemmy.ml.
Again, you should really stop revolving your entire life over one GitHub issue, and go touch grass.
You know what is easy to do, lie and make-up facts. It is so much easier to bullshit to be popular around here.
It's disgusting the people who lie like you do and believe liars: https://lemmy.ml/post/2421636
The irony in you saying that, and posting second-hand recounts by other people. They aren't closed. If they are closed, you wouldn't even be able to submit a registration application.
Man reading this thread, you're kind of a dumbass. Especially if you think rewording your answer here from the last reply to reframe to current time period vs what was being talked about would throw off the scent
They've not closed sign ups. Requiring approval of sign ups, is not closing sign ups. How am I wrong?
Currently no. But they had, which was clearly stated as it was talking on the past.
Show me in the code. Because I have closed the registration on my Lemmy server, and it does not turn off the "Sign Up" link or HTML input fields. But you sure like lies and deception
The developers of Lemmy seem to make every effort they can to avoid using Lemmy itself to discuss their [email protected] learning 101. They have made massive mistakes in SQL TRIGGER logic that they avoided to such a degree that their social motives are in question now. Github Issue 2910 was opened June 4, almost a month before the Reddit API deadline, and they ignored it. Just like they hang out on Matrix Chat and don't use Lemmy their own self to discuss code.
They have cultivated a kind of voodoo attitude towards PostgreSQL where people using Lemmy won't actually scrutinize the Rust code or PostgreSQL tuning parameters.
I mean having a bunch of new servers is not a problem. Just choose one that's been up for a while and more stable.
But each additional row in site_aggregates table was causing the instability itself. The SQL code had major flaws. Adding more servers actually made Lemmy crash more.