Which future of search?


Alex Iskold recently wrote on the ReadWriteWeb about potential improvements in search that could be derived from incorporating evidence from one social network to affect the ranking of documents. The idea is that people you know, people with similar interests, friends-of-friends, authorities, and “the crowd” could all contribute to change the ranking on documents that a search engine delivers to you because the opinions or interests of all these people can provide some information to help disambiguate queries.

These are the basic ideas that underlie recommender systems, with the additional twist of using social network evidence to identify “communities of searchers” (Freyne and Smyth, 2006) or trait-based similarity (Teevan et al, 2009). Authorities (“influencers”) are to some extent already incorporated in PageRank-type algorithms; Iskold’s proposal is to incorporate information from the Twitter feed into algorithms that assess the utility of found documents.

These sorts approaches are great tools for increasing the effectiveness of precision-based searches and for mitigating the negative effects of polysemy given a short query. Breakning news, celebrity gossip, and reference-finding are all good examples of searches that benefit from this “wisdom of crowds” approach.

But these techniques are limited in that they are good for finding things others have already found, rather than for exploring new ground. Exploratory search, on the other hand, has the goal of finding things that others have not yet found or making connections that have not yet been made. Thus these search tasks require algorithms that focus on recall and diversity, rather than on improved precision.

One way to achieve such goals is through a special kind of social search called collaborative search. The goal of collaborative search is to allow people with a shared information need (not just with shared social ties or interests) to work together to find information that may be more difficult or time-consuming for individuals to find on their own. Examples of domains where collaborative search may be more effective than recommendation-based social search include medical information seeking, intelligence analysis, academic research, certain kinds of travel planning, etc. Its strengths are bringing together people with different perspectives on a problem, and mediating their searches to allow the group to discover information that each individual may not have been able to find independently.