Can open source improve open reviewing?

on

Naboj is an overlay on arXiv.org that allows people to comment on articles, to rate articles, and (unlike SciRate.com) to rate the reviews as well. Unfortunately, the rather minimal interface does not make it easy to organize the display by highly-rated reviewers or by thoroughly-reviewed papers (i.e., papers with reviews that others found useful), or restrict search to particular domains.

These limitations are not inherent in the design of the review process or the data collected on the site, but rather are probably indicative of an under-resourced effort. I wonder if an open-source approach to the design of these kinds of tools would result in a more usable (and thus more useful) way of managing an open peer review process. Is open source the way to open reviewing? I would certainly consider contributing to it.

8 Comments

  1. Twitter Comment


    Posted “Can open source improve open reviewing?” [link to post]

    Posted using Chat Catcher

  2. Bojan Tunguz says:

    Thanks for mentioning Naboj on your blog. As you correctly point out, the limitations of Naboj are mostly due to limited resources with which I have had to contend during its development, mostly in terms of time. I would be very glad to get some more developers working with me on it, and hopefully as the word spreads about it I can come in contact with people who would like to collaborate on what I think is a very important project for the future of scientific publishing.

  3. Bojan, It would be interesting to explore the possibilities of making this into an open-source project.

  4. Scott Carter says:

    From the Chicago Boyz blog:

    In the end, it is the anonymous and secret nature of the peer review process that marks it as not part of actual science. The entire point of science is that all observers of a phenomenon can agree they see the same thing. Critical to creating that agreement is ruthless transparency. Secrecy is antithetical to the functioning of science, and peer review is a secret process. Science is not settled by the secret complaints of the anonymous.

    and:

    When scientists tell the public that a scientific study that used a large, custom-written piece of software has been “peer reviewed” does that mean the study faced the same level of peer scrutiny as did a study that used more traditional hardware instruments and procedures?

    Clearly not.

    A “ruthless transparency” would demand open software.

  5. It feels right to have openness serve both transparency and resource constraints.

  6. Gene, I am not sure that would really work. One of the main aims of Naboj is to have a centralized place where users can write comments and review scientific papers. Making it open-source would probably balkanize that aim.

  7. Gene Golovchinsky says:

    I am proposing open source development of one system, not many systems for the same purpose. As Scott commented above, openness of the system is appropriate for open reviewing.

    Tools and best practices for managing volunteer-based development projects are widely available. Pushing projects in that direction can be an effective mechanism for compensating for lack of resources.

  8. Well, I am not completely against the idea. If you have any links and suggestions for the best way of approaching this I would be very interested in learning more about it.

Comments are closed.