Journal of the American Medical Informatics Association, Vol 3, 422-428, Copyright © 1996 by American Medical Informatics Association

ARTICLES

Effects of a decision support system on the diagnostic accuracy of users: a preliminary report

AS Elstein, CP Friedman, FM Wolf, G Murphy, J Miller, P Fine, P Heckerling, T Miller, J Sisson, S Barlas, K Biolsi, M Ng, X Mei, T Franz and A Capitano
Department of Medical Education, University of Illinois at Chicago 60612, USA. aelstein@uic.edu

OBJECTIVES: To assess the effects of incomplete data upon the output of a computerized diagnostic decision support system (DSS), to assess the effects of using the system upon the diagnostic opinions of users, and to explore if these effects vary as a function of clinical experience. DESIGN: Experimental pilot study. Four clusters of nine cases each were constructed and equated for case difficulty. Definitive findings were omitted from the case abstracts. Subjects were randomly assigned to one of four clusters and were trained on the DSS prior to use. SUBJECTS: The study involved 16 physicians at three levels of clinical experience (six general internists, four residents in internal medicine, and six fourth-year medical students), from three academic medical centers. PROCEDURE: Each subject worked up nine cases, first without and then with ILIAD consultation. They were asked to offer up to six potential diagnoses and to list up to three steps that should be the next items in the diagnostic workup. Effects of DSS consultation were measured by changes in the position of the correct diagnosis in the lists of differential diagnoses, pre- and post-consultation. RESULTS: The DSS lists of diagnostic possibilities contained the correct diagnosis in 38% of cases, about midway between the levels of accuracy of residents and attending general internists. In over 70% of cases, the DSS output had no effect on the position of the correct diagnosis in the subjects' lists. The system's diagnostic accuracy was unaffected by the clinical experience of the users.

This article has been cited by other articles:

				E. Bottieau, J. Moreira, J. Clerinx, R. Colebunders, A. Van Gompel, and J. Van den Ende Evaluation of the GIDEON Expert Computer Program for the Diagnosis of Imported Febrile Illnesses Med Decis Making, June 1, 2008; 28(3): 435 - 442. [Abstract] [PDF]

				P. Ramnarayan, R. R. Kapoor, M. Coren, V. Nanduri, A. L. Tomlinson, P. M. Taylor, J. C. Wyatt, and J. F. Britto Measuring the Impact of Diagnostic Decision Support on the Quality of Clinical Decision Making: Development of a Reliable and Valid Composite Score J. Am. Med. Inform. Assoc., November 1, 2003; 10(6): 563 - 572. [Abstract] [Full Text] [PDF]

				E. S. Berner Diagnostic Decision Support Systems: How to Determine the Gold Standard? J. Am. Med. Inform. Assoc., November 1, 2003; 10(6): 608 - 610. [Full Text] [PDF]

				M A Graber and D VanScoy How well does decision support software perform in the emergency department? Emerg. Med. J., September 1, 2003; 20(5): 426 - 428. [Abstract] [Full Text] [PDF]

				H. S. F. Fraser, W. J. Long, and S. Naimi Evaluation of a Cardiac Diagnostic Program in a Typical Clinical Setting J. Am. Med. Inform. Assoc., July 1, 2003; 10(4): 373 - 381. [Abstract] [Full Text] [PDF]

				G. Hripcsak and A. Wilcox Reference Standards, Judges, and Comparison Subjects: Roles for Experts in Evaluating System Performance J. Am. Med. Inform. Assoc., January 1, 2002; 9(1): 1 - 15. [Abstract] [Full Text] [PDF]

				C. P. Friedman, A. S. Elstein, F. M. Wolf, G. C. Murphy, T. M. Franz, P. S. Heckerling, P. L. Fine, T. M. Miller, and V. Abraham Enhancement of Clinicians' Diagnostic Reasoning by Computer-Based Consultation: A Multisite Study of 2 Systems JAMA, November 17, 1999; 282(19): 1851 - 1856. [Abstract] [Full Text] [PDF]

				E. S. Berner, R. S. Maisiak, C. G. Cobbs, and O. D. Taunton Effects of a Decision Support System on Physicians' Diagnostic Performance J. Am. Med. Inform. Assoc., September 1, 1999; 6(5): 420 - 427. [Abstract] [Full Text] [PDF]

				E. S. Berner and R. S. Maisiak Influence of Case and Physician Characteristics on Perceptions of Decision Support Systems J. Am. Med. Inform. Assoc., September 1, 1999; 6(5): 428 - 434. [Abstract] [Full Text] [PDF]

				E. E. Westberg and R. A. Miller The Basis for Using the Internet to Support the Information Needs of Primary Care J. Am. Med. Inform. Assoc., January 1, 1999; 6(1): 6 - 25. [Abstract] [Full Text] [PDF]

				W. H. Hay, A. S. Elstein, C. P. Friedman, G. Murphy, and F. M. Wolf Computer-based Diagnostic Support Systems J. Am. Med. Inform. Assoc., May 1, 1997; 4(3): 256 - 256. [Full Text] [PDF]