This Article Full Text Full Text (PDF) All Versions of this Article: M1075v1 9/6/637    most recent Submit a response Alert me when this article is cited Alert me when eLetters are posted Alert me if a correction is posted Services Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Berrios, D. C. Articles by Fagan, L. M. Search for Related Content PubMed PubMed Citation Articles by Berrios, D. C. Articles by Fagan, L. M.

Methods for Semi-automated Indexing for High Precision Information Retrieval

Affiliations of the authors: Stanford Medical Informatics, Stanford University, Stanford, California.

Correspondence and reprints: Daniel C. Berrios, MD, PhD, MPH, Research Institute for Advanced Computer Science, Ames Research Center, NASA, MailStop 269-2, Moffett Field, CA94035; e-mail: <berrios{at}email.arc.nasa.gov>.

Objective. To evaluate a new system, ISAID (Internet-based Semi-automated Indexing of Documents), and to generate textbook indexes that are more detailed and more useful to readers.

Design. Pilot evaluation: simple, nonrandomized trial comparing ISAID with manual indexing methods. Methods evaluation: randomized, cross-over trial comparing three versions of ISAID and usability survey.

Participants. Pilot evaluation: two physicians. Methods evaluation: twelve physicians, each of whom used three different versions of the system for a total of 36 indexing sessions.

Measurements. Total index term tuples generated per document per minute (TPM), with and without adjustment for concordance with other subjects; inter-indexer consistency; ratings of the usability of the ISAID indexing system.

Results. Compared with manual methods, ISAID decreased indexing times greatly. Using three versions of ISAID, inter-indexer consistency ranged from 15% to 65% with a mean of 41%, 31%, and 40% for each of three documents. Subjects using the full version of ISAID were faster (average TPM: 5.6) and had higher rates of concordant index generation. There were substantial learning effects, despite our use of a training/run-in phase. Subjects using the full version of ISAID were much faster by the third indexing session (average TPM: 9.1). There was a statistically significant increase in three-subject concordant indexing rate using the full version of ISAID during the second indexing session (p < 0.05).

Summary. Users of the ISAID indexing system create complex, precise, and accurate indexing for full-text documents much faster than users of manual methods. Furthermore, the natural language processing methods that ISAID uses to suggest indexes contributes substantially to increased indexing speed and accuracy.

This article has been cited by other articles:

				Y. Huang, H. J. Lowe, D. Klein, and R. J. Cucina Improved Identification of Noun Phrases in Clinical Radiology Reports Using a High-Performance Statistical Natural Language Parser Augmented with the UMLS Specialist Lexicon J. Am. Med. Inform. Assoc., May 1, 2005; 12(3): 275 - 285. [Abstract] [Full Text] [PDF]

				Y. Huang, H. J. Lowe, and W. R. Hersh A Pilot Study of Contextual UMLS Indexing to Improve the Precision of Concept-based Representation in XML-structured Clinical Radiology Reports J. Am. Med. Inform. Assoc., November 1, 2003; 10(6): 580 - 587. [Abstract] [Full Text] [PDF]