Some weeks ago, I received an email from Stefano Mizzaro asking my opinion about his paper Quality Control in Scholarly Publishing: A New Proposal (pdf). In the meantime he came to Trento and we discussed face to face but I want to share here some quick comments I wrote on my wiki about the paper. I liked it, it is very clearly presented, it addresses a real problem and a more and more important one. The math is very clear, sound and makes sense. [Yes he found me because of the blog and not because of my papers and this keeps telling me something]. Read the comments to the paper.
Paper:
Quality Control in Scholarly Publishing: A New Proposal by Stefano Mizzaro
Abstract:
The Internet has fostered a faster, more interactive and effective model of scholarly publishing. However, as the quantity of information available is constantly increasing, its quality is threatened, since the traditional quality control mechanism of peer review is often not used (e.g., in online repositories of preprints, and by people publishing on their Web pages whatever they want).
This paper describes a new kind of electronic scholarly journal, in which the standard submission-review-publication process is replaced by a more sophisticated approach, based on judgments expressed by the readers: in this way, each reader is, potentially, a peer reviewer. New ingredients, not found in similar approaches, are that each reader’s judgment is weighted on the basis of the reader’s skills as a reviewer, and that readers are encouraged to express correct judgments by a feedback mechanism that estimates their own quality. The new electronic scholarly journal is described in both intuitive and formal ways. Its effectiveness is tested by several laboratory experiments that simulate what might happen if the system were deployed and used.