A small drunk goblin in every server throws a dart at a handwritten piece of paper with every post ever
No Stupid Questions
No such thing. Ask away!
!nostupidquestions is a community dedicated to being helpful and answering each others' questions on various topics.
The rules for posting and commenting, besides the rules defined here for lemmy.world, are as follows:
Rules (interactive)
Rule 1- All posts must be legitimate questions. All post titles must include a question.
All posts must be legitimate questions, and all post titles must include a question. Questions that are joke or trolling questions, memes, song lyrics as title, etc. are not allowed here. See Rule 6 for all exceptions.
Rule 2- Your question subject cannot be illegal or NSFW material.
Your question subject cannot be illegal or NSFW material. You will be warned first, banned second.
Rule 3- Do not seek mental, medical and professional help here.
Do not seek mental, medical and professional help here. Breaking this rule will not get you or your post removed, but it will put you at risk, and possibly in danger.
Rule 4- No self promotion or upvote-farming of any kind.
That's it.
Rule 5- No baiting or sealioning or promoting an agenda.
Questions which, instead of being of an innocuous nature, are specifically intended (based on reports and in the opinion of our crack moderation team) to bait users into ideological wars on charged political topics will be removed and the authors warned - or banned - depending on severity.
Rule 6- Regarding META posts and joke questions.
Provided it is about the community itself, you may post non-question posts using the [META] tag on your post title.
On fridays, you are allowed to post meme and troll questions, on the condition that it's in text format only, and conforms with our other rules. These posts MUST include the [NSQ Friday] tag in their title.
If you post a serious question on friday and are looking only for legitimate answers, then please include the [Serious] tag on your post. Irrelevant replies will then be removed by moderators.
Rule 7- You can't intentionally annoy, mock, or harass other members.
If you intentionally annoy, mock, harass, or discriminate against any individual member, you will be removed.
Likewise, if you are a member, sympathiser or a resemblant of a movement that is known to largely hate, mock, discriminate against, and/or want to take lives of a group of people, and you were provably vocal about your hate, then you will be banned on sight.
Rule 8- All comments should try to stay relevant to their parent content.
Rule 9- Reposts from other platforms are not allowed.
Let everyone have their own content.
Rule 10- Majority of bots aren't allowed to participate here.
Credits
Our breathtaking icon was bestowed upon us by @Cevilia!
The greatest banner of all time: by @TheOneWithTheHair!
Thank you Post Goblin, I love the work you do.
I think your prompt was wrong there mate. He said small not swol.
Looks more like an orc, maybe it's the goblins boss
Can I buy him a beer or something?
I hear he likes beans
This is my favorite theory so far.
Can confirm. Found the process on my server. I named him frank.
https://join-lemmy.org/docs/contributors/07-ranking-algo.html is everything you need
To summarize it for people that don't feel like clicking the link, it essentially takes the log of the post score and then divides it by an exponential function of the time since the post was published.
And this picture helps too: shows the decay in ranking scores for posts of different popularity (score) over time.
After a day or so, the curve flattens out. This probably explains why we keep seeing posts that are months old in "hot" - if not enough new material is being posted, after the first few pages of "hot", posts that are 5 days old and 5 months old are essentially the same due to the exponential decay function that was chosen.
That page gives this equation:
Rank = ScaleFactor * log(Max(1, 3 + Score)) / (Time + 2)^Gravity
Score = Upvotes - Downvotes
Time = time since submission (in hours)
Gravity = Decay gravity, 1.8 is default
My guess is that the "gravity" parameter is the issue at the moment. Something is needed to make the decay less steep, so that really old posts aren't making it up to the top of the feed.
There might be some way of tuning the gravity parameter dynamically based on how much content is being submitted, perhaps aiming for something like "the average age of the first 200 posts should be 10 days" (I made those numbers up, but the basic idea would be that the time decay should be steeper when lots of content is submitted and less steep when content is infrequent?)
I know some of those words.
don't worry, its already fixed. should be in the next release.
On my personal instance I'm running a build with that and its properly giving nice recent posts ( including the OP)
After all of this, I will amend my response to say that I think that there must be something going wrong with the algorithm. Consider these two consecutive posts on my "hot" feed:
The anti-vax nonsense from two years ago was appropriately downvoted to hell. The post right underneath it is one year old and has a post score of +13. Based on the equation above, the lower post must have a higher rank than the anti-vax post, as it should have both a higher numerator and a lower denominator.
Time for a review of the source code? Or am I missing something? Do other people see this phenomenon? No older, lower-scored post should be above a newer and higher-scored post in your feed, I think.
I don't think the max(1, 3 + score) work well neither, basically a post with a score of -2 and another of -100 have have the same log(1) = 0 rank.
Very nice analysis.
Maybe you want a more neutral and stable metrics for a dynamic measure of the gravity? Otherwise you can flood Lemmy with new posts to bury something.
Maybe something related to the average number of active users over the past 30 days over the topics you are looking at, which is harder to alter. But regardless, the steepness is definitely an issue as it should change with the number of posts.
Yeah - agreed. I don't know the best solution. The other issue is whether the algorithm is being applied to all feeds and communities in the same way. The experience will be quite different if browsing all on a highly federated, high activity instance, compared to just looking at your subscriptions or browsing a lower-activity single community. Maybe the answer is just in general to decrease the steepness of the curve.
So who can change the algorithm? Is it up to the admins of each instance (lemmy.world in my case) to change the numbers? There's not a centralized formula that each instance refers to is there?
Technically every admin could change it by making the change to the code and recompiling it. Practically it's just code contributors to lemmy development.
Damn. So comments are not included. Anything that has a crapton of comments yet is controversial won't be shown despite being hot.
To my understanding that's what "active" is for
Well it's doing a pretty shabby job then innit?
thanks for the explanation! I wonder whether it is possible, or rather scalable, if users can pick their own parameters, even define their own functions. Is this calculated and cached at the server side or user side?
I understand some of the words you said. Sounds like you are the person to fix it 😁.
Thank you.
Thanks 😊
And the relevant source code
And this is a great thing about open source software
Want to know how something works? Want to know the implications of something, or whether it is artificially manipulated? You can go directly to the code.
How does the algorithm work for other software, and is it authentic and not manipulated for other gains? Nobody knows except them, and bad stuff can be hidden away.
Can someone who knows PL/pgSQL help parse this line:
return floor(10000*log(greatest(1,score+3)) / power(((EXTRACT(EPOCH FROM (timezone('utc',now()) - published))/3600) + 2), 1.8))::integer;
It seems to me that the issue might be that the function returns an integer. If the scaling factor is inadequately large, then floor()
would return zero for tons of posts (any post where the equation inside floor()
evaluates to less than one). All of those posts would have equivalent ranks. This could explain why we start seeing randomly sorted old posts after a certain score threshold. Maybe better not to round here or dramatically increase the scaling factor?
I'm not sure what the units of the post age would be in here, though. Probably hours based on the division by 3600? And is log()
the natural log or base 10 by default?
In any case, something still must be going wrong. If I'm doing the math correctly, a post with a score of +25 should take approximately 203 hours (assuming log base 10) before it reaches a raw rank score of < 1 and gets floored to zero, joining all of the really old posts. So we should be seeing all posts from the last 8.5 days that had +25 scores before we see any of these really old posts... But that isn't what's happening.
Just curious, is this something that admins of individual instances could adjust for themselves? I could see some specialized instances being able to make use of a customized sorting algorithm for this.
If this is something that admins can adjust, does that impact anything with that content shared to or accessed from any federated instances?
I don't think any of the algorithm is expose to other instances so that wouldn't impact the communication between instances. At the end of the day this is open source so admins can freely build a forked version of Lemmy with a slightly different algorithm.
When you post, the Lemmy app secretly takes a photo of your face. This is then sent to a 3rd party AI application that looks at your facial features and ranks you on how hot you are. This is then sent back to the Lemmy server. This hotness score is then weighted by the users location that is viewing the feed (ie, an LA 7 might be a 9 in Chicago, or a 10 in alamaba if they are genetically related to you)
I too would like to know this and am too lazy to look at the source code. Maybe tomorrow.
"Maybe tomorrow" is about one fifth of the human condition
Only one fifth? Those are rookie numbers.
Have to leave 79% for "eh"
Whatever is being upvoted the most recently.
When it is not from you.
the hot feed is the same as reddits rising feed. its not a hot feed, its just random posts that have gotten an above average number of upvotes in the last hour. this site is next to impossible to use between the lack of a hot page and all of the niche communities not being populated thus making the front page and your subfeed the same 5 communities, and they're all memes.
Well, if it was a better fit, there’s always Return to Reddit.
Divining rods.
Perhaps, it's purpose is to help make old cold posts warmer by bringing them to the top.
Is that like a shit flambe?
It would be kind of cool to be able to define yourself the function that decides this for your account.