I’ve been doing a bit of thinking lately about search engines, algorithmic openness, and spammers. I suppose this was all prompted by a blog post recently on the Meaning of Open: http://googleblog.blogspot.com/2009/12/meaning-of-open.html
In this post, it is claimed that openness is good: open systems, open source, open data. This claim is held forth as true…for everything except for search algorithms. In the case of algorithms, the secret sauce must be kept exactly that: secret. Spammers would otherwise have too much power.
That claim makes me want to play around with a little thought experiment. What if the search algorithm were indeed fully open? What if everyone in the world knew exactly how rankings were done, and could modify their web pages so as to adapt themselves to whatever the ranking function is. In short, what if everyone were number one?
Well, my first thought is that no one could be number one for everything. If a page were too diverse, about too many different topics, or too long, it would likely already be discounted by the retrieval algorithm. So pages would need to stay focused. A ranking algorithm is, in a sense, a set of constraints. And no one page can simultaneously satisfy every single constraint. And some constraints are intrinsic, while some are extrinsic. Even though a webmaster has full control over those intrinsic aspects, and (through spammer bot networks) a modicum of control over some of the extrinsic aspects, the one thing that the webmaster cannot control is the user.
So it seems to me that if the ranking algorithm were completely open, what would start to happen is that more control over ranking, re-ranking, filtering, and exploring would start to fall to the user. Some of that control might be painful and unwanted: “just give me the home page for the Indiana Colts” some user might say, “don’t make me work at it!” But some of that control might be more beneficial — say to customize one’s ranking, or to mash up one’s results with friends and colleagues, so as to get an explicitly personalized view of the web. Third-party developers would start to appear and would offer solutions to end users to easily allow them to find the Colts’ home page where necessary, and still collaboratively plan vacations with their spouse, where necessary, too.
In short, if everyone were number one, then I would expect web search to start to become more decentralized, more like the web itself: Peer to peer, flexible, democratic yet built up into coalitions. And not controlled by a single or small group of entities.
What do you think? What would happen if everyone were number one?