This Article Full Text Full Text (PDF) Submit a response Alert me when this article is cited Alert me when eLetters are posted Alert me if a correction is posted Services Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Brown, P. J. B. Articles by Sönksen, P. Search for Related Content PubMed PubMed Citation Articles by Brown, P. J. B. Articles by Sönksen, P.

Evaluation of the Quality of Information Retrieval of Clinical Findings from a Computerized Patient Database Using a Semantic Terminological Model

Affiliations of the authors: The Guys', King's College, and St. Thomas' Hospitals Medical and Dental School, London, United Kingdom.

Correspondence and reprints: Philip J. B. Brown, MD, MRCGP, Hethersett Surgery, Great Melton Road, Hethersett, Norfolk NR9 3AB, United Kingdom; e-mail: <pjbb{at}hicomm.demon.co.uk>.

Abstract Objectives: To measure the strength of agreement between the concepts and records retrieved from a computerized patient database, in response to physician-derived questions, using a semantic terminological model for clinical findings with those concepts and records excerpted clinically by manual identification. The performance of the semantic terminological model is also compared with the more established retrieval methods of free-text search, ICD-10, and hierarchic retrieval.

Design: A clinical database (Diabeta) of 106,000 patient problem record entries containing 2,625 unique concepts in an clinical academic department was used to compare semantic, free-text, ICD-10, and hierarchic data retrieval against a gold standard in response to a battery of 47 clinical questions.

Measurements: The performance of concept and record retrieval expressed as mean detection rate, positive predictive value, Yates corrected and Mantel-Haenszel chi-squared values, and Cohen kappa value, with significance estimated using the Mann-Whitney test.

Results: The semantic terminological model used to retrieve clinically useful concepts from a patient database performed well and better than other methods, with a mean detection rate of 0.86, a positive predictive value of 0.96, a Yates corrected chi-squared value of 1,537, a Mantel-Haenszel chi-squared value of 19,302, and a Cohen kappa of 0.88. Results for record retrieval were even better, with a mean record detection rate of 0.94, a positive predictive value of 0.99, a Yates corrected chi-squared value of 94,774, a Mantel-Haenszel chi-squared value of 1,550,356, and a Cohen kappa value of 0.94. The mean detection rate, Yates corrected chi-squared value, and Cohen kappa value for semantic retrieval were significantly better than for the other methods.

Conclusion: The use of a semantic terminological model in this test scenario provides an effective framework for representing clinical finding concepts and their relationships. Although currently incomplete, the model supports improved information retrieval from a patient database in response to clinically relevant questions, when compared with alternative methods of analysis.

This article has been cited by other articles:

				S. B. Johnson, S. Bakken, D. Dine, S. Hyun, E. Mendonca, F. Morrison, T. Bright, T. Van Vleck, J. Wrenn, and P. Stetson An Electronic Health Record Based on Structured Narrative J. Am. Med. Inform. Assoc., January 1, 2008; 15(1): 54 - 64. [Abstract] [Full Text] [PDF]

				J. M. Green, J. R. Wilcke, J. Abbott, and L. P. Rees Development and Evaluation of Methods for Structured Recording of Heart Murmur Findings Using SNOMED-CT(R) Post-Coordination J. Am. Med. Inform. Assoc., May 1, 2006; 13(3): 321 - 333. [Abstract] [Full Text] [PDF]

				R. H. Dolin, L. Alschuler, S. Boyer, C. Beebe, F. M. Behlen, P. V. Biron, and A. Shabo (Shvo) HL7 Clinical Document Architecture, Release 2 J. Am. Med. Inform. Assoc., January 1, 2006; 13(1): 30 - 39. [Abstract] [Full Text] [PDF]

				J. Choi, M. L. Jenkins, J. J. Cimino, T. M. White, and S. Bakken Toward Semantic Interoperability in Home Health Care: Formally Representing OASIS Items for Integration into a Concept-oriented Terminology J. Am. Med. Inform. Assoc., July 1, 2005; 12(4): 410 - 417. [Abstract] [Full Text] [PDF]

				C. Duclos, G. L. Cartolano, M. Ghez, and A. Venot Structured Representation of the Pharmacodynamics Section of the Summary of Product Characteristics for Antibiotics: Application for Automated Extraction and Visualization of Their Antimicrobial Activity Spectra J. Am. Med. Inform. Assoc., July 1, 2004; 11(4): 285 - 293. [Abstract] [Full Text] [PDF]

				J.-I. Hwang, J. J. Cimino, and S. Bakken Integrating Nursing Diagnostic Concepts into the Medical Entities Dictionary Using the ISO Reference Terminology Model for Nursing Diagnosis J. Am. Med. Inform. Assoc., July 1, 2003; 10(4): 382 - 388. [Abstract] [Full Text] [PDF]

				G. Hripcsak and A. Wilcox Reference Standards, Judges, and Comparison Subjects: Roles for Experts in Evaluating System Performance J. Am. Med. Inform. Assoc., January 1, 2002; 9(1): 1 - 15. [Abstract] [Full Text] [PDF]

				R. H. Dolin, L. Alschuler, C. Beebe, P. V. Biron, S. L. Boyer, D. Essin, E. Kimber, T. Lincoln, and J. E. Mattison The HL7 Clinical Document Architecture J. Am. Med. Inform. Assoc., November 1, 2001; 8(6): 552 - 569. [Abstract] [Full Text] [PDF]

				A. Coenen, H. F. Marin, H.-A. Park, and S. Bakken Collaborative Efforts for Representing Nursing Concepts in Computer-based Systems: International Perspectives J. Am. Med. Inform. Assoc., May 1, 2001; 8(3): 202 - 211. [Abstract] [Full Text]