Collaborative Exploratory Search in the Wild

on

For quite a while now Jeremy and I have been characterizing collaborative exploratory search, and distinguishing it from other kinds of searches. While so far the exercise has been largely theoretical, and sometimes has been met with skepticism. Well, in Tuesday’s Washington Post, I found a passage that illustrates the kinds of activities we are talking about.

The article describes Sol Berkman, a man with a medical condition whose wife and son tried to determine whether the symptoms matched a particular disease.

She and Robert, who has written a book about research methods, conducted parallel Internet searches. Both agreed that Berkman seemed to display the classic triad of NPH symptoms: unsteady gait, progressive dementia and urinary incontinence. The problem is that each of these symptoms is commonly seen in patients with dozens of other disorders. Their hopes were buoyed when an MRI exam showed a buildup of fluid in the ventricles of Berkman’s brain, consistent with excess fluid.

It would have been great to observe and interview them as they performed their searches, as all detail of their activities is (rightly) omitted from the article. Well, almost all: we clearly see that they were working on the same problem (had a shared information need), that they were working in parallel (presumably because they did not have efficient tools to mediate the collaboration), and that their information need consisted of many aspects related to the various symptoms that Sol had. There were undoubtedly other symptoms associated with his condition that he did not exhibit. We also can assume that they had different degrees of experience with respect to online information seeking. We don’t know if they were working side-by-side, but my guess is that they worked independently and only shared their conclusions after the fact, or perhaps, at some intermediate stages during their search.

In all, it was a complex (but not atypical) information need shared by two people. In the end, they found enough evidence to pursue a course of action. Could they have done this better? How could their efficiency and effectiveness be improved?

Perhaps a system like SearchTogether might help them share results and avoid some duplication of effort. But a collaborative exploratory search system should help them do more: it should not only allow them to share results, but should use information found by one person to influence the results of the other. It should let the expert searcher guide the novice, but allow both to contribute relevance judgments equally.

In our study, we found that searchers working together found more useful documents, and were able to recognize useful documents better,  than searchers working in parallel. This sort of cross-searcher relevance feedback is just one way that deeply-mediated search can improve search effectiveness; other methods remain to be evaluated.

We’re a long way from a comprehensive solution to this class of problem, but it is hearteneing to find that our efforts will address real needs of real people.

7 Comments

  1. Great post Gene. I agree that this class of problems needs more attention, from both theoretical and tool design perspectives. SearchTogether would have helped these family members in their search for medical information. It does, to some extent, let information found by one person influence the results of others. Users can comment on useful Web pages and add these pages to the summary which is a list of useful links found by group members during the search. But the onus is on the user to read others comments and ratings on Web pages and make sense of the information found by them in order to direct her own search. And hence, I think UI-mediated collaborative exploratory systems need to provide additional means of enhancing participants sensemaking of the results found by other group members so they can direct their own search accordingly. (More on this at my CHI talk)

    In an algorithmically-mediated tool, I would think that the system would find documents based on feedback provided by other group members?

    Have you seen the study conducted by Joho et al. at IIiX 2008? They compared concurrent and independent search conditions in terms of strategies used by searchers. They found that in the concurrent condition participants actually avoided viewing documents viewed by other group members and hence didnt gain a complete understanding of the topic. This leads me to believe that instead of leaving it completely to the user to decide which Web pages visited by others are important (as in the case of SearchTogether), there needs to be some algorithmic-mediation of results.

  2. Thanks for pointing us to the Joho et al study. It sounds like an interesting set of conclusions, and I am looking forward to reading it. For the record, the paper can be found here.

    As far as algorithmic mediation, we have played with some algorithms based on rank fusion that combine evidence from multiple queries (issued by multiple people) to affect the ranking on new queries. That was our SIGIR paper.

  3. Gene–If you wanted any details about my searching on this, I’d be happy to oblige. Also, I’d like to learn more about your exploratory search work, as I am the editor of a newsletter for business information professionals (The Information Advisor:www.informationadvisor.com), on effective research techniques (readers are mostly business librarians) and may wish to cover this in an upcoming article.

  4. Sharoda, we also have another set of algorithms that we’ve played around with, not based on fusion. But we’re still in the process of publishing it.

    Robert, I don’t suppose you get the IEEE Computer Magazine? We have an introductory article in the March ’09 issue. Or, on a more technical level is the SIGIR paper Gene mentions above.

  5. Robert, Thanks for taking the time to comment here!

    Yes, it would be great if you could offer some details on how you searched. This kind of detail is invaluable when designing systems for real information needs (as opposed to made up ones that are so often used to motivate design). By documenting as many individual cases as practical, we hope to distill design recommendations for more effective and appropriate information seeking systems.

    Please do not hesitate to contact me by email (gene at fxpal dot com) if you’d like.

  6. Will do Gene–I am traveling for a bit but will be back in touch next week.
    Bob

  7. […] Collaborative Exploratory Search in the Wild […]

Comments are closed.