Timing thoughts

on

I’ve written about Google Instant before, but Daniel Tunkelang’s recent post triggered some additional reactions. Daniel writes that Instant is good because

Users spend less–and hopefully no time–in a limbo where they don’t know if the system has understood the information-seeking intent they have expressed as a query.

thus, the argument goes that by saving the user a few hundred milliseconds (and the need to press the Enter key), users will be better off because they will get feedback on the queries they run more quickly, and thus will be able to find the things that they are looking for more quickly.

I am not sure that the accountants and the psychologists would necessarily agree, in this case.

I agree with Daniel that responsive systems are better than slow ones, but it is much less clear to me that saving 300 milliseconds in the time to retrieve the first few document snippets makes a significant (or even noticeable) difference for most users. Sure, in the aggregate, Google saves tens of thousands of hours (or whatever). That time, however, is distributed over millions (or whatever) of searches. A typical Google search session consists of only a few queries, so saving the user even a few seconds in a task dominated by reading time is not likely to make any substantive difference.

Furthermore, it’s not the speed of thought of typing the query that matters, it’s the speed of thought of understanding the results. In common precision-oriented searches (searches that represent frequently-occurring information needs such as those for Bob Dylan’s lyrics), getting to the first batch of documents may be good enough. But in more complex situations typical of HCIR, the bulk of the time is spent reading and assessing documents, not typing the queries and waiting for the search engine.

How does the 300 millisecond delay of seeing the first ten documents compare with the time it takes to make sense of the results? I don’t have a definitive answer, but, for example, one study of cognitive processing reports that it took people well over 300 milliseconds to judge whether a presented number was less than or greater than 5 when that number was presented as a digit; if it was spelled out, the times rose to over 400 milliseconds. Since most document-related decision tasks are considerably more complex, I think obsessing over the time to deliver the first ten results is optimizing an aspect of the overall system that is mostly inconsequential.

I am all for responsive, dynamic interfaces for exploring document collections, but I think it’s a mistake to think that shaving 300 milliseconds off the time required to produce the first 10 documents will be important for HCIR. But it sure makes it easy to find out what Ju is up to.

2 Comments

  1. I agree that Google Instant offers a lot more utility for precision-oriented searches than for recall-oriented ones. But I think that, for web search, that’s a pretty big deal. At least in my view, today’s web search tools are primarily suited for precision-oriented searches. Recall-oriented search on the web is an open problem.

    And, to reiterate, I’m not concerned with clock time but rather the user’s perception of time. Perhaps it’s a stretch, but consider how the delay in a VoIP phone call affects a conversation. It’s only a fraction of a second, but the change is dramatic.

    As I hope I made clear in my post, I think there’s a wide space of unsolved HCIR problems that Google Instant doesn’t even begin to address. But I do think it’s a huge step forward for web search.

  2. I agree that timing can be important for “in the flow” kinds of interactions such as speaking, writing with a pen, or driving a car. I am not convinced, however, that web search is such an activity and that the advance is as huge as you believe. I am, however, concerned that giving people the results of the first matching suggested query automatically will reduce the diversity of results given the feedback accumulated via click-through data.

Comments are closed.