The TOCHI Option

on

Many people in the CHI community are aware of the range of problems associated with the CHI conference review process that tries to cram 1,300 or more submissions through a rather small reviewer pool with the goal of selecting the interesting and the important, while filtering out  inappropriate or unfinished work. Needless to say, the process is often imperfect.

There have been many laments and calls for change (e.g., here, here, here), and some recent positive changes in way to conference is run.

One change at CHI, now in its second year, is the federated model of reviewing. In this scheme, people submit papers to specific committees based on a description of topics that committee represents. Each committee then selects papers for presentation at the conference. The tighter thematic grouping is expected to improve the reliability of decisions, and the parallelization of the PC meeting streamlines that process as well. The exact impact of the federated approach to the CHI PC is not clear, however, and SIGCHI would should be encouraged to account for the effects of the change.

But these changes don’t materially affect the reviewing bottleneck, and thus don’t address the root problem of poor quality control in the acceptance process. A more promising direction is the streamlining of the ACM TOCHI review process (a three month turnaround is the current goal). Once a paper is accepted for publication in TOCHI, it is also eligible for presentation at the conference. This approach makes sense for several reasons:

  • It distributes the reviewing load over the entire year rather than concentrating it in one month
  • It is much more forgiving of incomplete but promising submissions by allowing authors to correct or expand the manuscript more efficiently than is possible in a conference time frame
  • It restores the journal’s role as the archival publication and the conference’s role as a networking and discussion venue

If the journal can succeed in implementing this fast turn-around, and if they can thereby divert some of the work previously submitted directly to the conferences. In addition to streamlining this process, TOCHI should publicize more widely the opportunity these changes present. While I was aware of the policy from discussions around CHI 2009 and from conversations with Shumin Zhai and Dan R. Olsen, the current and former TOCHI Chairs, I was not able to find any documentation of the policy on the TOCHI site or on the CHI 2010 site. I hope that the editors pay a bit more attention to the promotion of their otherwise fine efforts to improve the quality of SIG CHI publications.

3 Comments

  1. Twitter Comment


    Complaints abt review process at #CHI2010 taken seriously? @HCIR_GeneG blog post on the response [link to post] @acmtochi

    Posted using Chat Catcher

  2. Twitter Comment


    Posted “The TOCHI Option” [link to post] @acmtochi #chi2010

    Posted using Chat Catcher

  3. Shumin Zhai says:

    We are working hard to reduce the turn-around time for ToCHI. If you are asked to review for a journal, please respond promptly. It is okay to say no if your schedule does not allow a prompt review (along with recommendations of qualified reviewers you know is helpful). Say yes of course is preferred. Most researchers say yes most of the time particularly when the submission is in their area of expertise. If you say yes please review it right away if you can – your life does not get less busy :( Saying yes but drop the ball or wait for the action editor to nag repeatedly is one of the causes of journals’ long turn-around time. Please avoid that if you can.

    Thanks to many of you who have contributed to TOCHI — Shumin

Comments are closed.