Fediverse

32101 readers

1206 users here now

A community to talk about the Fediverse and all it's related services using ActivityPub (Mastodon, Lemmy, KBin, etc).

If you wanted to get help with moderating your own community then head over to [email protected]!

Rules

Posts must be on topic.
Be respectful of others.
Cite the sources used for graphs and other statistics.
Follow the general Lemmy.world rules.

Learn more at these websites: Join The Fediverse Wiki, Fediverse.info, Wikipedia Page, The Federation Info (Stats), FediDB (Stats), Sub Rehab (Reddit Migration)

founded 2 years ago

MODERATORS

[email protected]

What is going on behind the scenes when searching for a lemmy post in a search engine? (lemmy.world)

submitted 2 days ago by [email protected] to c/[email protected]

11 comments fedilink hide all child comments

I was looking for some collection of posts earlier about Proton Mail and the whole controversy with the CEO, and I opened a post the lemmy instance that was suggested was lemmy.zip but the community and the poster were from lemmy.world so that made me ask myself a bunch of questions. Reference link

Note: I used duckduckgo

Here are some questions I have:

How does the search engine decide which instance to link you to as you could in theory show every instance for the same post?
Could you get a result where all the results are the same post just different instances?
Do you think that could deter new people finding out about lemmy through search results?
How can an instance make themselves more visible in the search results (for exposure)?
I did not get any results from lemmy clients such as vger.app the only results were direct instances, will this always be the case?

I remember learning about search engines a while back but I don't know how relevant that information is any more. Having crawlers and the more a website is linked in other websites the higher up in the search result will be and the whole robot.txt thing.

I know if I wanted to search for something specific in lemmy I could just use its own search function, but what about people who ask general questions and that happens to be answered in a lemmy post. I wanted to know how exposed we are/ will be to people who don't yet know about lemmy.

you are viewing a single comment's thread
view the rest of the comments

[–] gigachad 14 points 2 days ago

Instances that disagree with being found in search engines are not shown. Instance admins can configure their robots.txt by adding lemmy-search. All other instances can theoretically be found. I think their priority depends on the laws of SEO (Search Engine Optimization). This probably means that a post on myownlemmy1337 that is federated with lemmy.world, will be found as a post on lemmy.world.

So, if Lemmy was very famouse, I guess it's possible to get pages over pages with the same result from different instances. However search engines usually have a way to exclude "similar" results.

For voyager it may be possible, they do not want to be found, I don't know about this though. You could add site:vger.app to your search prompt for testing this.