this post was submitted on 28 Dec 2023
64 points (89.0% liked)
Fediverse
28223 readers
421 users here now
A community to talk about the Fediverse and all it's related services using ActivityPub (Mastodon, Lemmy, KBin, etc).
If you wanted to get help with moderating your own community then head over to [email protected]!
Rules
- Posts must be on topic.
- Be respectful of others.
- Cite the sources used for graphs and other statistics.
- Follow the general Lemmy.world rules.
Learn more at these websites: Join The Fediverse Wiki, Fediverse.info, Wikipedia Page, The Federation Info (Stats), FediDB (Stats), Sub Rehab (Reddit Migration), Search Lemmy
founded 1 year ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
No, not as simply as that. That's the basic idea of recommendation systems that were common in the 1990s. The algorithm requires a tremendous amount of dimensionality reduction to work at scale. In that simple description it would need a trillion weights to compare the preferences of a million users to a million other users. If you reduce it to some standard 100-1000ish dimensions of preference it becomes feasible, but at the low end only contains about as much information as your own choices about subscribed to or blocked communities (obviously it has a much lower barrier of entry).
There's another important aspect of learning that the simple description leaves out, which is exploration. It will quickly start showing you things you reliably like, but won't experiment with things it doesn't know you'd like or not to find out.
Why would this be the case? It shows you stuff that people who like similar stuff that you do like, but people have diverse interests so wouldn't it be likely that the people that like one thing like other things that you hadn't known about and that leads to a form of guided exploration?
There's two problems. The first is that those other things you might like will be rated lower than things you appear to certainly like. That's the "easy" problem and has solutions where a learning agent is forced to prefer exploring new options over sticking to preferences to some degree, but becomes difficult when you no longer know what is explored or unexplored due to some abstraction like dimension reduction or some practical limitation like a human can't explore all of Lemmy like a robot in a maze.
The second is that you might have preferences that other people who like the same things you've already indicated a taste for tend to dislike. For example there may be other people who like both Boba and Cofee but people who like one or the other tend to dislike the other. If you happen to encounter Boba first then Cofee will be predicted to be disliked based on the overall preferences of people who agree with your Boba preference.
With this specific algorithm, I don't necessarily think that would be the case. It only shows you fewer links from people who like the links that you dislike. It doesn't show you fewer links based on what people who are like you dislike which is what it seems like you are describing.
Also, it doesn't have to be this specific algorithm that we implement but I thought the idea was unique so I thought I'd share it anyway.
It seems to be working well enough for me now so I plan to keep using it and see what it's like.
Whether or not you use downvotes doesn't really matter.
If what you like is well represented by the Boba drinkers and the Boba drinkers disproportionally don't like Cofee then Cofee will be disproportionally excluded from the top of your results. Unless you explore deeper the Cofee results will be pushed to the bottom of your results. And any that happen to come to the top will have arrived there from broad appeal and will have very little contribution to thinking you like Cofee.
If you don't let the math effectively push things away that are disliked by the people who like similar things as you then everything will saturate at maximum appeal and the whole system does nothing.