This Article Full Text Full Text (PDF) Submit a response Alert me when this article is cited Alert me when eLetters are posted Alert me if a correction is posted Services Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Kuilboer, M. M. Articles by van Bemmel, J. H. Search for Related Content PubMed PubMed Citation Articles by Kuilboer, M. M. Articles by van Bemmel, J. H.

Simulating an Integrated Critiquing System

Affiliations of the authors: Department of Medical Informatics, Erasmus University Rotterdam (MMK, JvdL, JHvB); Sophia Children's Hospital Rotterdam (JCdJ); Dijkzigt Academic Hospital Rotterdam (SEO); and Department of Family Practice, Erasmus University Rotterdam (BP); all in The Netherlands.

Correspondence and reprint requests to: M. M. Kuilboer, MD, Erasmus University Rotterdam Ee2100, PO Box 1738, 3000 DR Rotterdam, The Netherlands. e-mail: <kuilboer{at}mi.fgg.eur.nl>.

Abstract Objective: To investigate factors that determine the feasibility and effectiveness of a critiquing system for asthma/COPD that will be integrated with a general practitioner's (GP's) information system.

Design: A simulation study. Four reviewers, playing the role of the computer, generated critiquing comments and requests for additional information on six electronic medical records of patients with asthma/COPD. Three GPs who treated the patients, playing users, assessed the comments and provided missing information when requested. The GPs were asked why requested missing information was unavailable. The reviewers reevaluated their comments after receiving requested missing information.

Measurements: Descriptions of the number and nature of critiquing comments and requests for missing information. Assessment by the GPs of the critiquing comments in terms of agreement with each comment and judgment of its relevance, both on a five-point scale. Analysis of causes for the (un-)availability of requested missing information. Assessment of the impact of missing information on the generation of critiquing comments.

Results: Four reviewers provided 74 critiquing comments on 87 visits in six medical records. Most were about prescriptions (n = 28) and the GPs' workplans (n = 27). The GPs valued comments about diagnostics the most. The correlation between the GPs' agreement and relevance scores was 0.65. However, the GPs' agreements with prescription comments (complete disagreement, 31.3%; disagreement, 20.0%; neutral, 13.8%; agreement, 17.5%; complete agreement, 17.5%) differed from their judgments of these comments' relevance (completely irrelevant, 9.0%; irrelevant, 24.4%; neutral, 24.4%; relevant, 32.1%; completely relevant, 10.3%). The GPs were able to provide answers to 64% of the 90 requests for missing information. Reasons available information had not been recorded were: the GPs had not recorded the information explicitly; they had assumed it to be common knowledge; it was available elsewhere in the record. Reasons information was unavailable were: the decision had been made by another; the GP had not recorded the information. The reviewers left 74% of the comments unchanged after receiving requested missing information.

Conclusion: Human reviewers can generate comments based on information currently available in electronic medical records of patients with asthma/COPD. The GPs valued comments regarding the diagnostic process the most. Although they judged prescription comments relevant, they often strongly disagreed with them, a discrepancy that poses a challenge for the presentation of critiquing comments for the future critiquing system. Requested additional information that was provided by the GPs led to few changes. Therefore, as system developers faced with the decision to build an integrated, non-inquisitive or an inquisitive critiquing system, the authors choose the former.

This article has been cited by other articles:

				D. L. Sanders and D. Aronsky Biomedical Informatics Applications for Asthma Care: A Systematic Review J. Am. Med. Inform. Assoc., July 1, 2006; 13(4): 418 - 427. [Abstract] [Full Text] [PDF]

				M. M. Kuilboer, M. A. M. van Wijk, M. Mosseveld, and J. van der Lei AsthmaCritic: Issues in Designing a Noninquisitive Critiquing System for Daily Practice J. Am. Med. Inform. Assoc., September 1, 2003; 10(5): 419 - 424. [Abstract] [Full Text] [PDF]

				M. M Kuilboer, M. A. van Wijk, M. Mosseveld, E. van der Does, B. P Ponsioen, J. C de Jongste, S. E Overbeek, and J. van der Lei Feasibility of AsthmaCritic, a decision-support system for asthma and COPD which generates patient-specific feedback on routinely recorded data in general practice Fam. Pract., October 1, 2002; 19(5): 442 - 447. [Abstract] [Full Text] [PDF]