This Article Full Text Full Text (PDF) All Versions of this Article: M1670v1 12/3/322    most recent Submit a response Alert me when this article is cited Alert me when eLetters are posted Alert me if a correction is posted Services Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via Google Scholar Google Scholar Articles by Rothschild, A. S. Articles by Lehmann, H. P. Search for Related Content PubMed PubMed Citation Articles by Rothschild, A. S. Articles by Lehmann, H. P.

Information Retrieval Performance of Probabilistically Generated, Problem-Specific Computerized Provider Order Entry Pick-Lists: A Pilot Study

Affiliation of the authors: Division of Health Sciences Informatics, Johns Hopkins University School of Medicine, Baltimore, MD.

Correspondence and reprints: Adam S. Rothschild, MD, Department of Biomedical Informatics, Columbia University, Vanderbilt Clinic 5th Floor, 622 West 168th Street, New York, NY 10032; e-mail: <adam.rothschild{at}dbmi.columbia.edu>.

Received for publication: 08/11/04; accepted for publication: 01/24/05.

Objective: The aim of this study was to preliminarily determine the feasibility of probabilistically generating problem-specific computerized provider order entry (CPOE) pick-lists from a database of explicitly linked orders and problems from actual clinical cases.

Design: In a pilot retrospective validation, physicians reviewed internal medicine cases consisting of the admission history and physical examination and orders placed using CPOE during the first 24 hours after admission. They created coded problem lists and linked orders from individual cases to the problem for which they were most indicated. Problem-specific order pick-lists were generated by including a given order in a pick-list if the probability of linkage of order and problem (PLOP) equaled or exceeded a specified threshold. PLOP for a given linked order-problem pair was computed as its prevalence among the other cases in the experiment with the given problem. The orders that the reviewer linked to a given problem instance served as the reference standard to evaluate its system-generated pick-list.

Measurements: Recall, precision, and length of the pick-lists.

Results: Average recall reached a maximum of .67 with a precision of .17 and pick-list length of 31.22 at a PLOP threshold of 0. Average precision reached a maximum of .73 with a recall of .09 and pick-list length of .42 at a PLOP threshold of .9. Recall varied inversely with precision in classic information retrieval behavior.

Conclusion: We preliminarily conclude that it is feasible to generate problem-specific CPOE pick-lists probabilistically from a database of explicitly linked orders and problems. Further research is necessary to determine the usefulness of this approach in real-world settings.