I'm kind of curious to understand how they're blocking other search engines. I was under the impression that search engines just viewed the same pages we do to search through, and the only way to 'hide' things from them was to not have them publicly available. Is this something that other search engines could choose to circumvent if they decided to?
this post was submitted on 26 Jul 2024
662 points (97.4% liked)
Technology
59581 readers
3407 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related content.
- Be excellent to each another!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, to ask if your bot can be added please contact us.
- Check for duplicates before posting, duplicates may be removed
Approved Bots
founded 1 year ago
MODERATORS
Couldn't a search engine just aggregate the result from Google, filter the Reddit responses, and then add those results to their own organic results?
They're also blocking posts by users who aren't banned or even got a warning. It appears to the user as though it's been posted, but it hasn't.
load more comments
(5 replies)
Hot take here.
I do believe in free information.
Instead of investing money in stop crawlers why do not make the data they are trying to crawl available to everyone for free so we can have a better world all together?
load more comments
(2 replies)
It's still possible to search with "site:reddit.com ..."
Has it been implemented yet or are they blocking non-flagged searches? Which seems odd.
load more comments
(2 replies)