Extremely Unofficial CHI 2010 review survey

on

Yesterday, Lennart Nacke expressed the desire to act on the suggestion in a blog post I wrote to review the reviewers. So why not? I would like to see if we can collect some data to inform the debate about obtaining quality reviews for conferences such as CHI. The goal is to see if the availability of authors’ ratings on reviews of papers can be used to improve the reviewer selection process and to give direct feedback to reviewers as well.

This is not an official survey administered by CHI, but rather a bit of guerrilla data collection to help explore possibilities. Once the official CHI 2010 results are sent out, I would love to hear from you. If you were an author of a CHI 2010 submission, and would like to participate, here is what I’d like to know:

  1. The submission number that was assigned to you.
  2. The initials of the corresponding author or the name of the institution (a sanity check to cut down on the amount of spam). While we don’t have access to the submission metadata, we might be able to use this information later to clean the data. The goal, however, is to keep this analysis as anonymous as possible.
  3. For each review that you received (typically R1-R5, but sometimes more), we would like to get your opinion on the quality of the review provided on a 1-5 scale (5 is best). Did the review assess the paper’s contribution correctly? Was the review helpful in identifying problems with the paper?
  4. Was the paper accepted?
  5. Comments

9 Comments

  1. Twitter Comment


    Posted “Extremely Unofficial CHI 2010 review survey” [link to post] #chi2010

    Posted using Chat Catcher

  2. Twitter Comment


    RT @HCIR_GeneG: Posted “Extremely Unofficial CHI 2010 review survey” [link to post] #chi2010

    Posted using Chat Catcher

  3. Twitter Comment


    RT @HCIR_GeneG: Posted “Extremely Unofficial CHI 2010 review survey” [link to post] #chi2010

    Posted using Chat Catcher

  4. Twitter Comment


    RT @HCIR_GeneG Posted “Extremely Unofficial CHI 2010 review survey” [link to post] #chi2010

    Posted using Chat Catcher

  5. Twitter Comment


    @HCIR_GeneG :) Thank you for the extremely unofficial CHI 2010 review survey at your blog: [link to post] #chi2010

    Posted using Chat Catcher

  6. Twitter Comment


    @whazelwo Extremely Unofficial CHI 2010 review survey [link to post]

    Posted using Chat Catcher

  7. Twitter Comment


    @jonfroehlich thks for RT Are people filling out the form, or just looking? Seeing traffic but no responses [link to post]

    Posted using Chat Catcher

  8. Twitter Comment


    RT @jonfroehlich: RT @HCIR_GeneG Posted “Extremely Unofficial CHI 2010 review survey” [link to post] #chi2010

    Posted using Chat Catcher

  9. Twitter Comment


    submitted to #chi2010? review the reviewers! [link to post] (@HCIR_GeneG)

    Posted using Chat Catcher

Comments are closed.