Communicating about Collaboration

on

What does it mean to collaborate while searching?

There are many different ways to characterize collaborative information seeking, many dimensions on which collaborative search systems can be categorized.

For the past few years Jeremy Pickens and I have been thinking that our model of collaborative exploratory search needs some further explication. Or maybe we’re just trying to understand it better ourselves. We have found that to explain what our model is, we have to simultaneously explain what our model is not.  This has led to numerous discussions not only about the various dimensions of collaboration, but also about the relative importance among those dimensions for distinguishing between systems.

We talked about some of these characterizations in our HCIR’07 paper, where we described two dimensions: synchrony and intent.  We extended this discussion  in our JCDL 2008 workshop position paper, where we introduced depth of mediation and location. But we found there still remained some confusion about the exact nature of these dimensions, particularly with respect to synchrony.

Our plan is to come up with a few blog posts that explain the dimensions in more detail, and to engage with our readers (we assume we have readers) to work out the kinks in the model and in our exposition.

Briefly, the current version of the model consists of the following dimensions:  intent represents the degree to which people share an explicitly-articulated information need. Synchrony describes how influence flows among collaborators. Depth of mediation is the extent to which the contributions of different people are taken into consideration when performing search results. Finally, location describes whether searchers are co-located or distributed.

In the following weeks we’ll discuss these dimensions in more detail, and will update this post to point to these upcoming discussions.

25 Comments

  1. It’s interesting that you describe synchrony as the influence that flows among collaborators. So if a team is equally influential with each other, they will be more synchronous?

    I have been thinking about the benefits of synchronous vs. asynchronous communication in a slightly different way (although it might contribute to “influence flows”). Face-to-face interactions might cause higher, denser information exchanges between collaborators, which might result in a better matching of new information (facts) to cognitive processing speed. One example would be brainstorming where people can bounce ideas off each other in real time, ponder and reflect on those idea as they crop up. An asynchronous interaction (with a time delay) would make this type of brainstorming more difficult, though it wouldn’t necessarily preclude it. More likely, asynchronous exchanges would change the flow of conversation to support other types of activities. Maybe here’s how it directs the flow of influence among collaborators? I might imagine that it would support fewer cognitive activities around search, but it might serve as a convenient platform for simple information exchange. (I have no empirical evidence one way or another on this yet, though!)

    Being co-located or distributed surely influences how synchronous or asynchronous interactions can be. Are collaborators more likely to have “high” intent, synchrony, and depth of mediation if they are co-located? I’m particularly interested in how an individual searcher exploits the social resources in his environment (intent would not be equal among all people), and how this scenario plays out differently when his co-participants are co-located versus remote/distributed.

  2. We tried to make the dimensions are independent as possible, and so they tend to model different aspects of the complete human-machine system. For example, intent is purely a behavioral/cognitive (i.e., human) dimension, whereas depth of mediation characterizes algorithms that are involved in supporting information seeking, and the extent to which these algorithms represent each person’s contributions explicitly. Thus collab filtering is deeply-mediated because each person’s judgments or behaviors are recorded and identified.

    Synchrony was the term we used at first, but you are right that it doesn’t quite get at what we’re after. That’s why we started talking in terms of influence. This gets away from connotations of strict temporal correlation, but still captures the back-and-forth nature of collaborative activity.

    Co-location is interesting because it can affect out-of-channel communication (e.g., voice, non-verbal communication, etc.) and because it can reflect the design of the system. Some systems permit distributed collaboration, whereas others require co-location.

  3. Brynn — Yes, the synchronicity issue is a tricky one. We should hopefully have a blog post up in two weeks that goes into it in a little more detail, and tries to pull in examples and themes from other research as well.

    For now, I’ll just note that the dictionary has 5 different definitions of synchronicity. Let me quote the first and the fifth (from m-w.com): (1) happening, existing, or arising at precisely the same time, and (5) of, used in, or being digital communication (as between computers) in which a common timing signal is established that dictates when individual bits can be transmitted and which allows for very high rates of data transfer.

    So the way we’ve been thinking about synchronicity is in the “common timing signal” way. Replace “common timing signal”, with “common relevance / feedback / activity signal”, aka a “common influence signal”. With that definition, we can say that any bits of information (queries, unexamined results, relevance-marked results, query suggestions, etc.) that arise from User 1 and flow into the information retrieval system (either the UI part of the system or the deeper algorithm part of the system) are immediately and completely “synchronized” to any and all current information seeking activities of User 2. User 1’s interactions with the collaborative information retrieval system are essentially “filtered” in real-time by User 2’s actions, and vice versa. Again, it’s a “common timing/influence signal” type of synchronization.

  4. This discussion is awesome! :)

    @Jeremy – Liked your explanation of synchronicity as “any bits of information that arise from User 1 and flow into the IR system are immediately and completely synchronized to current information seeking activities of User 2″. I realized that this definition is not mutually exclusive with definition (1) from m-w.com. In other words, this kind of synchronicity can be supported even in “asynchronous” search where group members do not search at the same time, but all their actions with the IR system are saved and used to filter the information seen by other users.

    For example, SearchTogether supports this kind of synchronicity (as per definition 2) for users who collaborate both synchronous and asynchronously (as per definition 1). Last summer Merrie and I conducted user studies with SearchTogether where we designed the study such that 3 members of a group searched “synchronously” (at the same time, but in different physical locations) and a fourth group member logged into the same search session later in time “asynchronously”. For group members who searched synchronously, all their actions with the tool (such as web pages they found, comments and ratings on web pages etc.) were dissipated to all other group members in real time. When the fourth group member logged in asynchronously, they could see the saved actions of the other group members. From a UI-perspective, we found that there were several challenges for the fourth group member to make sense of the information found by the other three who had searched synchronously. I think this might apply to algorithmically mediated results too.

    I’ll be presenting results from these studies at CHI this year, so if you guys are going to be there I’d love to discuss this more!

  5. Glad you like the discussion! :-) It’s good for me, too.

    With the idea of synchronicity as a common timing (influence) signal, between the endpoint users, rather than absolute same-time activity of the endpoint users themselves, I think you can start to play faster and looser with the actual amount of time that passes between each user action. Because the point is that it is the “system bus”, the common timing signal, that is synchronous, rather than each user.

    Some types of collaborative search might yield faster real-time round-trip flows of information between users — minutes, seconds, or even milliseconds if you’re talking about colocated work on a shared table display. Some types might yield slower round-trip flows, minutes, hours, or maybe even days. But as long as the flow is still active, as long as there is that “synchronous” shared bus, common timing signal, then (I feel) it should be characterized as synchronous collaboration. There might be some point at which you stretch the notion so far that it snaps. I certainly cannot imagine synchronicity with year-long gaps between round trips. But the main point is that there *is* a round trip. Is that your understanding, too?

    It sounds like, in your experiments, the 4th user really was asynchronous, because there was no round-trip common timing bus between that 4th user and the three previous users. Influence flowed unidirectionally, from the 3-users to the 4th user. It never flowed back, in the other direction. Right? I would very much call this an asynchronous collaboration. Heck, even if the 4th user had been working at the exact same time as the three other users, but influence had only been traveling from the 3 to the 4th, I would call that asynchronous. Yes? No?

  6. […] is Part 2 of (at least) 5 in a series of posts about Collaborative Information Seeking.  Part 1 is found […]

  7. @Jeremy – I agree with your idea of synchronicity as a common time signal between end-point users. From an implementation perspective that is how collaborative search systems would work. For instance, for UI-mediated systems, you would synch collaborators’ UIs at fixed intervals of time so that all the information being generated by the group is reflected in each UI. What I’m not sure about is how long this interval of time needs to be for the collaboration to still be considered synchronous. For instance, if this interval of time can be of the order of days, then would we consider the 4th user in my study to be synchronous? I don’t consider that user to be synchronous, which means I’m bounding the value of that interval to seconds or milliseconds.

    In our experiments the influence only flowed between 3-users to the 4th user because that is how we constructed the task. However, all the information generated by the 4th user (web pages, comments, ratings) was stored in the system and we could have conducted a third round of experiments in which the 3-users logged in back to see what the 4th user had found (we leave this to future work, of course :)) and continued the search task. So the influence flowed both ways because information generated by all users was stored in the system and would be reflected in all UIs whenever the users logged back in to the search session of the group. Does that make sense?

  8. Our take on synchronicity and influence is that if one user has generated some information about the shared information need, that information should be available immediately to other collaborating users. This symmetrical use of information is what creates collaboration. The time scale of notifications depends on the time scale of activity: We could have a real-time session during which two or more people work at the same time, or more of a turn-taking approach, where one person finds a document, then the other person absorbs that information and does something else, etc. What’s important is that they both have the opportunity to influence each other through the sharing of knowledge (to use Colum Foley’s term). This is in contrast to recommender systems in which the people whose behaviors or opinions were aggregated to produce a recommendation are not affected by (or even necessarily aware of) those people who use the recommendations.

    In your case, it sounds like the system allowed the potential for symmetric collaboration, but the way the study was set up, that information was not used in the UI.

  9. For instance, if this interval of time can be of the order of days, then would we consider the 4th user in my study to be synchronous?

    Yes, if the 3 users come back and then built on what the 4th user did, as part of a shared task, then I would definitely call that synchronous. The influence would flow from the 3, to the 4th, and back to the 3. There would be a shared, bidirectional bus.

    If, however, the 3 were satisfied with their information seeking activities, and never came back to that information-need-based task (effectively declaring their activities over and done), then that is an asynchronous collaboration.

    Thinking about this in terms of the common/shared timing bus: If the 3 users have disconnected themselves from the bus, by declaring their satisfaction/completion of the task, then influence is no longer flowing bidirectionally between them and the 4th user. The 4th user is asynchronously collaborating with the 3.

  10. All this elaboration suggests that synchrony is the wrong name. Should we talk about symmetry of influence instead?

  11. Symmetric vs. asymmetric collaboration, rather than synchronous vs. asynchronous? I think that’s a surface that we all need to scratch more — there is probably something to it.

    Still, I have to slightly disagree with Sharoda about bounding the interval to seconds or even milliseconds. The interval that I am talking about is not the frequency or clock speed of the influence bus. I agree, that bus should be humming along at the millisecond rate.

    Rather, I am talking about the frequency with which any one collaborating user places another “atomic information retrieval data unit” onto the bus. That unit could be a query, a relevance judgment, a set of documents (e.g. from a ranked list), etc. Those actions don’t happen every millisecond, or even every second. Sometimes a user needs to spend 30 seconds, or 5 minutes, reading a document before she or he makes a relevance judgment. Sometimes, one of the users gets up for a 15-minute coffee break or hour-long lunch break. Anything that his or her partner does during this time away from the keyboard should immediately be synchronized to the back-end search engine. But just because one user walks away for a few minutes or even an hour doesn’t mean that they’re not working on a shared information need search task “simultaneously” or “synchronously”.

    I think about it in terms of whether the users are “in sync” with each other, versus “out of sync”. If Sharoda’s 3 users are done with the task, and are never coming back to it, after any length of time, they are out of sync with the 4th user. But if they do come back, and keep going on the same task, they are in sync.

  12. I thought a little more: One reason why I don’t like symmetric and asymmetric is that you can have both symmetric and asymmetric synchronous influences. In the domain expert/search expert collaboration scenario, the types of influence that each person exerts on each other is not necessarily symmetric. How each influences the other is based on the roles that each is playing in the system. But they can still asymmetrically synchronously influence each other.

  13. I had intended asymmetry to reflect the presence rather than the quality of information flow between people. Clearly different roles will cause different kinds of information to flow.

  14. Ah, yes, I get it now. Yes, symmetric/asymmetric influence is good. But doesn’t the notion of being “in sync” and “out of sync” with someone capture or communicate the same idea? Maybe it should be synchronous and non-synchronous, rather than asynchronous.

  15. […] is the third post by Jeremy and me in a series on collaborative information seeking. The first was an introduction to the space, and the second dealt with the topic of collaborative intent. This […]

  16. […] judgments of relevance (i.e., which documents were “bookmarked”).  In terms of our model of collaborative information seeking, their system had synchronized data, UI-level mediation, and, of course, explicitly shared […]

  17. […] (although not guaranteed) to interact with people around the information they post. And of course explicit collaboration is a possible technique for finding information with other […]

  18. […] to the workshop organization and to the writing of the report. Finally, I am delighted that collaborative information seeking is featured as an important aspect of the field. We hope that this report will inspire others to […]

  19. […] seems like a great opportunity to implement a collaborative search interface that would mediate the collaboration between the people handling the phone calls and the […]

  20. […] to contrast these theories with our assumptions of more focused collaboration that underlie collaborative search. When Hertzum and Haque argue that effort is required to maintain collaborative teams, we assume […]

  21. […] Alex Iskold recently wrote on the ReadWriteWeb about potential improvements in search that could be derived from incorporating evidence from one social network to affect the ranking of documents. The idea is that people you know, people with similar interests, friends-of-friends, authorities, and “the crowd” could all contribute to change the ranking on documents that a search engine delivers to you because the opinions or interests of all these people can provide some information to help disambiguate queries. […]

  22. […] can produce radically different effects. It also works well as an analogy to (you guessed it) collaborative search. Social search based on recommendations, whether inferred from user behavior or from expressed […]

  23. […] and I have been blogging about collaborative search for a while, and it is our pleasure to announce that Merrie Morris and we are organizing another […]

  24. […] problems, due in no small part to the upcoming HCIR conference and the CFP for the 2nd Workshop on collaborative IR.  But a really clear, high-level articulation of the key factors in HCIR are laid out in Daniel […]

  25. […] the past couple of years, we have developed some novel information retrieval algorithms such as collaborative search. While we have evaluated the work in various ways (e.g., evaluating algorithms offline and testing […]

Comments are closed.