This Article Full Text Full Text (PDF) Submit a response Alert me when this article is cited Alert me when eLetters are posted Alert me if a correction is posted Services Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Aronsky, D. Articles by Haug, P. J. Search for Related Content PubMed PubMed Citation Articles by Aronsky, D. Articles by Haug, P. J.

Evaluation of a Computerized Diagnostic Decision Support System for Patients with Pneumonia

Study Design Considerations

Affiliations of the authors: Vanderbilt University, Nashville, Tennessee (DA); LDS Hospital, University of Utah, Salt Lake City, Utah (KJC, PJH).

Correspondence and reprints: Dominik Aronsky, MD, PhD, Department of Biomedical Informatics, Informatics Center, Eskind Biomedical Library, Vanderbilt University, 2209 Garland Avenue, Nashville, TN 37232-8340; e-mail: dominik.aronsky{at}mcmail.vanderbilt.edu.

Received for publication: 10/05/00; accepted for publication: 05/07/01.

Planning the clinical evaluation of a computerized decision support system requires a strategy that encompasses the different aspects of the clinical problem, the technical difficulties of software and hardware integration and implementation, the behavioral aspects of the targeted users, and the discip of study design. Although clinical information systems are becoming more widely available, only a few decision support systems have been formally evaluated in clinical environments. Published accounts of difficulties associated with the clinical evaluation of decision support systems remain scarce. The authors report on a variety of behavioral, logistical, technical, clinical, cost, and work flow issues that they had to address when choosing a study design for a clinical trial for the evaluation of an integrated, real-time decision support system for the automatic identification of patients likely to have pneumonia in an emergency department. In the absence of a true gold standard, they show how they created a credible, clinically acceptable, and economical reference standard for the diagnosis of pneumonia, to determine the overall accuracy of the system. For the creation of a reference standard, they describe the importance of recognizing verification bias and avoiding it. Finally, advantages and disadvantages of different study designs are explored with respect to the targeted users and the clinical setting.

This article has been cited by other articles:

				D. Aronsky, P. J. Haug, C. Lagor, and N. C. Dean Accuracy of Administrative Data for Identifying Patients With Pneumonia American Journal of Medical Quality, November 1, 2005; 20(6): 319 - 328. [Abstract] [PDF]

				K. Kawamoto, C. A Houlihan, E A. Balas, and D. F Lobach Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features critical to success BMJ, April 2, 2005; 330(7494): 765. [Abstract] [Full Text] [PDF]

				D. Aronsky, E. Kasworm, J. A. Jacobson, P. J. Haug, and N. C. Dean Electronic Screening of Dictated Reports to Identify Patients with Do-Not-Resuscitate Status J. Am. Med. Inform. Assoc., September 1, 2004; 11(5): 403 - 409. [Abstract] [Full Text] [PDF]