10 sats \ 0 replies \ @brugeman OP 6 Dec 2022 \ parent \ on: Building a new Web Search Engine. Just for you, Stackers! bitcoin
That's a great question, I've thought about it, and I don't really have the answer (
There exists a distributed search engine, it's called YaCy. Here is their public demo web UI (no need to setup the client): https://yacy.searchlab.eu/
As far as I can see, a decade of work on this thing, and it's search results are still really bad. Even my toy project seems to do better. Why is that? I'm not sure.
My guess is that the most important thing with search is fighting abuse (spam, etc). With many servers operated by independent entities, each having their own view on quality or shilling different sites, getting consistently good results might be a really big problem.
I'd say we should start with our goals. Why decentralize it?
- To avoid censorship - that's a very broad idea, must dig very deep into what, who, how etc.
- To get personalization without surveillance - that can be done with a local app (like a browser extension), that will do query-rewriting and post-filtering of the non-personalized search result on the client, without decentralizing the engine.
I agree this is an attractive direction to think about, but at this point I'm not ready to invest in experiments there :)