This Article Full Text Full Text (PDF) Appendix 1 and Table 1 All Versions of this Article: M1062v1 10/5/512    most recent Submit a response Alert me when this article is cited Alert me when eLetters are posted Alert me if a correction is posted Services Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via Google Scholar Google Scholar Articles by Ammenwerth, E. Articles by Kulikowski, C. Search for Related Content PubMed PubMed Citation Articles by Ammenwerth, E. Articles by Kulikowski, C.

Developing and Evaluating Criteria to Help Reviewers of Biomedical Informatics Manuscripts

Affiliations of the authors: Research Group Assessment of Health Information Systems, University for Health Informatics and Technology Tyrol (UMIT), Innsbruck, Austria (EA); Department of Medical Informatics, University of Heidelberg, Germany (ACW, PK, SS); Institute for Biostatistics and Documentation, University of Innsbruck, Austria (HU); Erasmus University, Rotterdam, The Netherlands (JHV); National Library of Medicine, Bethesda, Maryland (ATM); Institute for Health Information Systems, University for Health Informatics and Technology Tyrol, Innsbruck (UMIT), Austria (RH); Department of Computer Science, Rutgers, The State University of New Jersey, New Brunswick, New Jersey (CK), USA.

Correspondence and reprints: Asst.-Prof. Dr. Elske Ammenwerth, Research Group Assessment of Health Information Systems, University for Health Informatics and Technology Tyrol (UMIT), Innrain 98, 6020 Innsbruck, Austria; e-mail: <elske.ammenwerth{at}umit.at>.

Received for publication: 11/21/01; accepted for publication: 05/06/03.

Peer-reviewed publication of scientific research results represents the most important means of their communication. The authors have annually reviewed a large heterogeneous set of papers to produce the International Medical Informatics Association (IMIA) Yearbook of Medical Informatics. To support an objective and high-quality review process, the authors attempted to provide reviewers with a set of refined quality criteria, comprised of 80 general criteria and an additional 60 criteria for specific types of manuscripts. Authors conducted a randomized controlled trial, with 18 reviewers, to evaluate application of the refined criteria on review outcomes. Whereas the trial found that reviewers applying the criteria graded papers more strictly (lower overall scores), and that junior reviewers appreciated the availability of the criteria, there was no overall change in the interrater variability in reviewing the manuscripts. The authors describe their experience as a "case report" and provide a reference to the refined quality review criteria without claiming that the criteria represent a validated instrument for quantitative quality measurement.