Many people in the CHI community are aware of the range of problems associated with the CHI conference review process that tries to cram 1,300 or more submissions through a rather small reviewer pool with the goal of selecting the interesting and the important, while filtering out inappropriate or unfinished work. Needless to say, the process is often imperfect.
One change at CHI, now in its second year, is the federated model of reviewing. In this scheme, people submit papers to specific committees based on a description of topics that committee represents. Each committee then selects papers for presentation at the conference. The tighter thematic grouping is expected to improve the reliability of decisions, and the parallelization of the PC meeting streamlines that process as well. The exact impact of the federated approach to the CHI PC is not clear, however, and SIGCHI would should be encouraged to account for the effects of the change.
But these changes don’t materially affect the reviewing bottleneck, and thus don’t address the root problem of poor quality control in the acceptance process. A more promising direction is the streamlining of the ACM TOCHI review process (a three month turnaround is the current goal). Once a paper is accepted for publication in TOCHI, it is also eligible for presentation at the conference. This approach makes sense for several reasons:
- It distributes the reviewing load over the entire year rather than concentrating it in one month
- It is much more forgiving of incomplete but promising submissions by allowing authors to correct or expand the manuscript more efficiently than is possible in a conference time frame
- It restores the journal’s role as the archival publication and the conference’s role as a networking and discussion venue
If the journal can succeed in implementing this fast turn-around, and if they can thereby divert some of the work previously submitted directly to the conferences. In addition to streamlining this process, TOCHI should publicize more widely the opportunity these changes present. While I was aware of the policy from discussions around CHI 2009 and from conversations with Shumin Zhai and Dan R. Olsen, the current and former TOCHI Chairs, I was not able to find any documentation of the policy on the TOCHI site or on the CHI 2010 site. I hope that the editors pay a bit more attention to the promotion of their otherwise fine efforts to improve the quality of SIG CHI publications.