You are right. On the one hand, it's kind of bad, naive distributed architecture (my day job), it could have been done much better. On the other hand, the more important point is that it demonstrates an alternative to centralized. We'll learn a lot about usage patterns here, get new ideas, and either improve Lemmy or build something better from the ground up. Big thanks to Reddit for driving users this way to test scalability and get much better knowledge of usage.
Selfhosted
A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.
Rules:
-
Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.
-
No spam posting.
-
Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it's not obvious why your post topic revolves around selfhosting, please include details to make it clear.
-
Don't duplicate the full text of your blog or github here. Just post the link for folks to click.
-
Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).
-
No trolling.
Resources:
- awesome-selfhosted software
- awesome-sysadmin resources
- Self-Hosted Podcast from Jupiter Broadcasting
Any issues on the community? Report it using the report flag.
Questions? DM the mods!
It's not distributed architecture as you normally think it - it's a decentralised federation. It's an important distinction from your typical distributed architecture app.
What makes a distributed system good that Lemmy hasn't done? Seems like a pretty robust system to me, seems like scaling issues are on the instance host themself. With Reddit's experience, I don't see how there are issues
A network of (“thousands of”) servers has — like most things — pros and cons.
Some of the pros are:
- The network is more resiliant against outages. If lemmy.ml is down, all other users can still access the network.
- It's hard to take legal action against the network or to buy it out (like Big Players™ like to do to get rid of potential competitors).
- It allows various similar or even conflicting moderation policies. The network, i.e. the infrastructure doesn't allow or prohibit any specific opinion (the communities do).
- It allows for different ways to pay the bills: goodwill of the admin, donaitions, ads, fee or selfhosting. The latter also allows great control over the data so you control your privacy.
Some of the cons are:
- Content is replicated across servers, which increases the total amount of data stored.
- Latency and speed suffer.
- Interoperability with the wider Fediverse is less than 100%, which can create confusion and frustration.
- Discovery is more difficult.
I just commented on this in another thread: https://lemmy.world/comment/76011
TL;DR: The server-to-client interactions on Lemmy are a lot heavier than the server-to-server interactions, so even if you're just using your own server to interact with communities on other servers, it should still take load off of the servers you would have been using directly.
I have my own Lemmy instance running on my home server, but I'm here. "But Bizzle," you may be asking yourself, "why go through all the trouble of configuring your own instance just to wind up on Lemmy.World anyway?"
I'm glad you asked! And the answer is that federation only fetches parent comments. I'm glad Lemmy exists, and I'm going to keep using it, but we need federated sibling comments for this to actually be good, in my opinion.
EDIT: I actually couldn't have been more wrong.
I would be happy if my locally posted comment showed up on lemmy.world at all :-) (If this one does, maybe I was just impatient with the initial sync or so)
EDIT: Nevermind, I was too impatient :-)
I can attest sibling comments are meant to be federated (example thread from another instance: https://lemmy.world/comment/97687), but I was here at the very start of this instance and there were federation issues. Posts not showing, comments not showing on other instances. It's all very new so there may be technical issues.
I'm not sure I understand what you're saying. Did you mean that child comments are not federated?
I don't think that's correct.
Seeing as we're viewing this from many different instances, it certainly sounds like a configuration issue on their side
Every server just has a cache … there is no profit for the whole network …
I wouldn't say that caching is no profit. Yesterday there were several times when lemmy.ml was struggling or effectively down for some people, but despite complaints over there I could read lemmy.ml communities just fine through my instance. Caching meant that I was isolated from the service interruption, and the lemmy.ml server was isolated from my contribution to its load.
I'm quite worried of how well this federation system will work in the long run, especially when more people coming from Rexxit. As people make more post/comments, every federated instance will have to cache more redundant contents from each other, which also will use more storage thus increasing the fee of every instance hoster. There's also another problem of visibility in search engines. Because Lemmy/Kbin can be hosted by anyone, it makes searching on a specific domain impossible, unlike how I can just add "reddit" in the search query. Also since there are multiple Lemmy/Kbin instances, there's a chance there'll be similar communities spread over, fragmenting the communities even further. Until they can find a way to fix those problem, I don't think federation is suited for large scale communities.
As for fragmentation problem, maybe adding a global search for communities like this will help reducing fragmentation. Users can still make their own community in their instance, while other people who don't need to can easily find the community they want.
After a day of use, I'm incredibly disappointed.
The fragmentation problems, and lack of cohesive community discovery (or even apparently any agreed standards for sharing communities etc. across instances in a way the most popular app can reliably recognise as being a community and not an external link or mailto address) will make Lemmy an absolute non-starter for 99% of potential users.
I'm sure there are solutions, but as it stands I can't see Lemmy gaining any widespread adoption without a significant leap in user friendliness in regard to how federated instances are implemented and managed.
I don't see fragmentation as a problem at all. The number of total subscribers is published when doing a search and is the ultimate primary consolidator just like reddit. There were many redundant subs on reddit for any given subject, they just had no patronage. The process of establishing primacy takes time. Three days ago .world had less than 1k users and all of Lemmy had less active users than half those present on any one of several instances right now. The .world instance is 10 days old.
The priorities of decentralized service will not align with the antithesis model. I see a minor complexity barrier to entry as a positive filter for some of the worst quality users.
Agreed. If decentralized doesn't appeal to certain people then this isn't for them. I came from reddit. I'm not trying to make this into reddit.
I haven't even been using it for day, and I share your disappointment. However, I understand that Lemmy is in its infancy. There are huge UX hurdles to overcome, and it's a lot for two developers to carry. The hope is that more devs will join, and make a good UX -- For what it's worth, the UI is quite neat IMHO, it's just the UX with regards to federation and discoverability.
Having a ways to add instances and then replicate community lists would be a start. Having to manually fiddle with URLs of other communities is weird.
What there needs to be is concerted development focus on fixing these quality of life issues. Unfortunately, there was not much time allowed for this to happen seeing as it was about a week or two from the announcement to the start of the blackout. These things take time and development time isn't always available.
Redundancy is also happening on centralized servers. Also it is text so I would not be too concerned.
I've created my own instance in order to not create more load on others and it took a minute to realise I needed to populate it myself, would be nice to have a default view aggregating popular posts etc. across instances. But maybe I'm just asking for too much hehe
That's an interesting idea. Maybe you could even choose the "default subs" for your instance from across lemmy.
What's the alternative? You go full-banana decentralised or mega-site Reddit. I think Lemmy is a nice middle ground
This has definitely been a problem with communities being created on the bigger instances and not utilising smaller instances. Happy for someone to say I'm wrong etc, but I think there would be merit in capping instances to x number of users or communities, to force the user base to spread out.
Also, the way signups work, (ie you find a community you like then click sign up but that signs you up to that instance), further exacerbates the issue and the confusion around how federation works. The sign up links on each instance should lead either to a page with an instance finder, or to a random instance that matches the profile of, and is already federated with, the instance you were on. Otherwise the larger instances have a monopoly and are just going to lead to a bad user experience when they can't cope with the traffic.
It's a self defeating prophecy if users only want to sign up to the instances with the big communities, because then everyone is going to keep creating communities there and nobody is going to want to join a smaller instance.
I might be talking nonsense and am happy to be told why that is all wrong :)
Yes, there should be instance caps, and they should be visible to users.
That way users can scale, choose, without much thinking.
This same techinque works everywhere, for example MMO games. You have availability visible and choose servers according to it.
This would fix scaling partially without much technical changes.
I don't think that there are thousands. The fediverse stat's show about 300 servers, 200 or so made in the last week.
At that rate, it is not too bad. I expect there will be a plateau at some point, relatively soon, where the need for new ones stop, and the experimental ones disappear.
It's a bit worse than that actually. I'm now seeing several communities with exactly the same name that originate on different servers - so clearly Lemmy doesn't have a rule about duplication once you cross a server boundary. That's going to get unwieldy quite fast particularly if, I dunno, "Aww" gets popular on two separate servers at the same time - I guess I'll have to subscribe to both...
Well one instance shouldn't monopolize a community. If it takes a dump on one instance at least it exists elsewhere. If I want to start up my own cat community I don't see why that's an issue.
I agree, I don't particularly see this as an actual issue... Nothing stops you from subscribing to both.
Just like there could be a [email protected] and a [email protected]. Nobody is confused with emails when it comes to this... The difference is that it's slightly more work than reddit because r/aww is one particular thing and it's assumed we're talking about Reddit because of it's unique format. Here it's just c/aww on lemmy.ml, but that's a bit of the point of the [email protected] structure of naming.
I LOVE that there's [email protected] and [email protected]. Different communities ran by different groups will end up with different content. Then I can shop for the content I want myself.
Nobody can singularly own the name. I always found that to be a big problem on reddit. r/trees comes to mind, if there was an actual arborist community that want r/trees, well they were fucked. And that's kind of jacked. This way it doesn't matter. Just pick a different instance that doesn't already have c/trees and post there... or better, start your own instance to host it.
I don't know... in the future people could even start up instances of lemmy on domains like lemmy.jobs, lemmy.help or lemmy.hobby to aggregate major communities based on topics. lemmy.jobs for instance could be an instance that houses professional the arborist and the domain would make it clear the intent. Or even better... drop the lemmy all together and register jobs.social or similarly descriptive domain names.
I know we're all a hodge-podge of domains now because a lot of us are just spinning up instance on domains we already have... but the potential is there.
This problem existed on reddit too still. You have r/games r/game r/gamers r/gamenews r/gamernews etc. All trying to do the exact same thing.
I think this comment convinced me. Because you're right, on Reddit there were always offshoot communities that were essentially the same exact thing just of different sizes and run by different people. There'll probably always be the "most popular" one, and then several offshoots for the same topic but perhaps a better sense of community because it's hundreds or thousands of users vs millions or tens of millions of users.
Remembering the exact instance and community name combinations will take a little extra effort, but not significantly and subscribing negates that mostly.
r/trees comes to mind, if there was an actual arborist community that want r/trees, well they were fucked.
There was. They ended up with, I think, /r/marijuana_enthusiasts or something like that. It was quite funny to both sides, at least it was like 15 years ago.
This is a feature, not a bug. But we definitely need a solution to make subscribing/coalescing them easier for users. Mastodon allows subscribing to topics (hashtags) - I think something similar is needed here, but that will evolve naturally over time.
I don't get argument about duplicates. The same situation was on reddit - you've got few, sometimes more, subs about same topic. You could subscribe to whichever you wanted. Why on Lemmy this is suddenly a problem?
I think users are still having trouble with the mental model for browsing Lemmy.
The first interaction with the service is already fragmented - you need to choose where to create an account and start browsing. Even though you can browse communities from other servers, people are now seeing them through the lens of "fragmented" "my server vs other server" and that creates the illusion that these duplicates are somehow a huge issue.
But duplicates can actually be quite useful - a community called "memes" on Lemmy.world could attend to a different audience than a community also called "Memes" but made in an instance entirely in French.
Also, if two instances have two communities you enjoy, with the same name... Subscribe to both? Nothing stops you from doing that. It's okay. Reddit had "me_irl" and "meirl" which were the exact same, but with different mods, a relatively similar number of subscribers and quite honestly the same content. I didn't know the actual difference between the two, and I still do not know - I just subscribed to both and kept getting depressing memes to cry before going to sleep. No issues.
Are there ways to manage lists of such? For example, on the former platform that doesn't deserve a call out, you can do "me_irl+meirl" and aggregate both into one feed. This makes reading the (albeit potentially cross posted) content in a unified feed much easier.
Another similar point I'm having a hard time getting over is that with a centralized platform, it is easy to go to "Subject A", and see everything on that subject. However, now I need to see "Subject [email protected]", "Subject [email protected]", "Subject [email protected]"... Yes, I could subscribe to them all, but this ultimately end up creating a noisy home feed with also "Subject [email protected]", "Subject [email protected]", "Subject [email protected]", "Subject [email protected]", ... etc. all baked into one feed, as opposed to just something focused on "Subject A".
Lastly, discoverability leaves a lot of room for desire. Today, I'm fairly new to Lemmy, I am actively seeking out communities that I might be interested in, across multiple popular instances, and hoping that federation is enabled between the two instances. Tomorrow, I'd find that I'm subscribed to too many (see the noisy main feed issue above), and I'd remove a bunch. Next week, am I likely to go to the Join Lemmy directory to find new instances, and add "duplicate" communities from newly popular instances? I think not.
I think the long term survival of the platform (to expand beyond just us tech nerds that hate the former platform) will depend a lot on streamlining this workflow to make content discovery much more consistent. Even a simple option where a pseudo "!Community@" (with no instance) feed that aggregates all the "!Community" regardless of instance that you've subscribed to, might go a long way.
Discovery really has been the biggest drawback for me. The r/system combined with wikis and sidebars made it very easy to find interesting things.
That's lacking in lemmy so far. Which, it isn't a bad thing, barriers to entry have benefits. But from a user perspective, trying to replace reddit, the difficulty in navigating and finding things is frustrating.
But I'm coming from reddit, and they aren't meant to be the same. The issues are part of what makes it next to impossible for what happened there to happen in a federated system. And I'm so fucking sick of corporate bullshit ruining good things . I figure that lemmy will catch up in feature parity soon enough, and there's bound to be apps that make it easier to use at some point.
I just wish I had the resources to run a server myself.
That's a really good analogy. Still, there needs to be an easier way to search remote communities. Copy pasting community links in search bar is really clunky.
They are not useless, if the users would actually spread out among them. Each server has its limits.
Sounds like you would prefer a peer2peer approach instead of federation? A few years ago, I wrote up why I don’t belief in p2p. In short: It’s too costly and too complex.
Are all these thousands of lemmy servers useless?
almost. It's actually worse than that - when you subscribe to a community from your server it will fetch like 20 posts and that's it, you'll get only new stuff after that, so there's no possibility to do a full mirror of selfhosted, for example, if you started your instance today and didn't fetch posts and comments manually.
ActivityPub per se is just a spec on s2s/s2c communication, which is not a great thing since in many cases it assumes single source of truth, which potentially puts huge load on more popular instances.
I think a quick and dirty hack to this could be the following - each linked instance may maintain cache of announces (so there would be benefit of just forwarding original http signed requests w/o being afraid of malicious actor), which your instance could pull, this way you could populate your mirror without overloading the original source.
Distributed activities propagation though... Let's say there are some design steps involved to make this truly distributed, however I feel like it's possible.
Privacy wise for me it is more convenient to run my own instance and have my own private communities.
every instance is sharing in the traffic to browse the fediverse. Not one service is responsible for serving content, you (the instance admin) are only serving for your members.
The downside of this is there is a huge amount of replicated data stored everywhere. Content of popular communities will be scraped by and stored on many many servers, filling up servers and increasing storage and bandwith bills for all those servers