This Article Full Text Full Text (PDF) Supplemental Data Submit a response Alert me when this article is cited Alert me when eLetters are posted Alert me if a correction is posted Services Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Nadkarni, P. Articles by Brandt, C. Search for Related Content PubMed PubMed Citation Articles by Nadkarni, P. Articles by Brandt, C.

UMLS Concept Indexing for Production Databases

A Feasibility Study

Affiliation of authors: Yale University School of Medicine, New Haven, Connecticut.

Correspondence and reprints: Prakash M. Nadkarni, MD, Center for Medical Informatics, Yale University School of Medicine, P.O. Box 208009, New Haven, CT 06520-8009; e-mail: <Prakash.Nadkarni{at}yale.edu>.

Objectives: To explore the feasibility of using the National Library of Medicine's Unified Medical Language System (UMLS) Metathesaurus as the basis for a computational strategy to identify concepts in medical narrative text preparatory to indexing. To quantitatively evaluate this strategy in terms of true positives, false positives (spuriously identified concepts) and false negatives (concepts missed by the identification process).

Methods: Using the 1999 UMLS Metathesaurus, the authors processed a training set of 100 documents (50 discharge summaries, 50 surgical notes) with a concept-identification program, whose output was manually analyzed. They flagged concepts that were erroneously identified and added new concepts that were not identified by the program, recording the reason for failure in such cases. After several refinements to both their algorithm and the UMLS subset on which it operated, they deployed the program on a test set of 24 documents (12 of each kind).

Results: Of 8,745 matches in the training set, 7,227 (82.6 percent ) were true positives, whereas of 1,701 matches in the test set, 1,298 (76.3 percent) were true positives. Matches other than true positive indicated potential problems in production-mode concept indexing. Examples of causes of problems were redundant concepts in the UMLS, homonyms, acronyms, abbreviations and elisions, concepts that were missing from the UMLS, proper names, and spelling errors.

Conclusions: The error rate was too high for concept indexing to be the only production-mode means of preprocessing medical narrative. Considerable curation needs to be performed to define a UMLS subset that is suitable for concept matching.

This article has been cited by other articles:

				Y. Huang and H. J. Lowe A Novel Hybrid Approach to Automated Negation Detection in Clinical Radiology Reports J. Am. Med. Inform. Assoc., May 1, 2007; 14(3): 304 - 311. [Abstract] [Full Text] [PDF]

				B. Hazlehurst, H. R. Frost, D. F. Sittig, and V. J. Stevens MediClass: A System for Detecting and Classifying Encounter-based Clinical Events in Any Electronic Medical Record J. Am. Med. Inform. Assoc., September 1, 2005; 12(5): 517 - 529. [Abstract] [Full Text] [PDF]

				Y. Huang, H. J. Lowe, D. Klein, and R. J. Cucina Improved Identification of Noun Phrases in Clinical Radiology Reports Using a High-Performance Statistical Natural Language Parser Augmented with the UMLS Specialist Lexicon J. Am. Med. Inform. Assoc., May 1, 2005; 12(3): 275 - 285. [Abstract] [Full Text] [PDF]

				L. Marenco, T.-Y. Wang, G. Shepherd, P. L. Miller, and P. Nadkarni QIS: A Framework for Biomedical Database Federation J. Am. Med. Inform. Assoc., November 1, 2004; 11(6): 523 - 534. [Abstract] [Full Text] [PDF]

				C. Friedman, L. Shagina, Y. Lussier, and G. Hripcsak Automated Encoding of Clinical Documents Based on Natural Language Processing J. Am. Med. Inform. Assoc., September 1, 2004; 11(5): 392 - 402. [Abstract] [Full Text] [PDF]

				J. C. Denny, J. D. Smithers, R. A. Miller, and A. Spickard III "Understanding" Medical School Curriculum Content Using KnowledgeMap J. Am. Med. Inform. Assoc., July 1, 2003; 10(4): 351 - 362. [Abstract] [Full Text] [PDF]

				J. M. Fisk, P. Mutalik, F. W. Levin, J. Erdos, C. Taylor, and P. Nadkarni Integrating Query of Relational and Textual Data in Clinical Databases: A Case Study J. Am. Med. Inform. Assoc., January 1, 2003; 10(1): 21 - 38. [Abstract] [Full Text] [PDF]

				H. Liu, S. B. Johnson, and C. Friedman Automatic Resolution of Ambiguous Terms Based on Machine Learning and Conceptual Relations in the UMLS J. Am. Med. Inform. Assoc., November 1, 2002; 9(6): 621 - 636. [Abstract] [Full Text] [PDF]

				P. G. Mutalik, A. Deshpande, and P. M. Nadkarni Use of General-purpose Negation Detection to Augment Concept Indexing of Medical Documents: A Quantitative Study Using the UMLS J. Am. Med. Inform. Assoc., November 1, 2001; 8(6): 598 - 609. [Abstract] [Full Text] [PDF]

				F. S. McDonald, P. L. Elkin, and P. Nadkarni UMLS Concept Indexing for Production Databases: A Feasibility Study J. Am. Med. Inform. Assoc., September 1, 2001; 8(5): 512 - 515. [Full Text] [PDF]