This Article Full Text Full Text (PDF) All Versions of this Article: M1124v1 9/6/653    most recent Submit a response Alert me when this article is cited Alert me when eLetters are posted Alert me if a correction is posted Services Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Bachmann, L. M. Articles by ter Riet, G. Search for Related Content PubMed PubMed Citation Articles by Bachmann, L. M. Articles by ter Riet, G.

Identifying Diagnostic Studies in MEDLINE: Reducing the Number Needed to Read

Affiliations of the authors: University of Zürich, Zürich, Switzerland (LMB, RC, PE); University of Amsterdam (GtR).

Correspondence and reprints: Lucas M. Bachmann, MD, 1 Horten Centre, University of Zürich, Bolleystrasse 40, Postfach Nord, CH-8091 Zürich, Switzerland; e-mail: <lucas.bachmann{at}evimed.ch>.

Objectives. The search filters in PubMed have become a cornerstone in information retrieval in evidence-based practice. However, the filter for diagnostic studies is not fully satisfactory, because sensitive searches have low precision. The objective of this study was to construct and validate better search strategies to identify diagnostic articles recorded on MEDLINE with special emphasis on precision.

Design. A comparative, retrospective analysis was conducted. Four medical journals were hand-searched for diagnostic studies published in 1989 and 1994. Four other journals were hand-searched for 1999. The three sets of studies identified were used as gold standards. A new search strategy was constructed and tested using the 1989-subset of studies and validated in both the 1994 and 1999 subsets. We identified candidate text words for search strategies using a word frequency analysis of the abstracts. According to the frequency of identified terms, searches were run for each term independently. The sensitivity, precision, and number needed to read (1/precision) of every candidate term were calculated. Terms with the highest sensitivity x precision product were used as free text terms in combination with the MeSH term "SENSITIVITY AND SPECIFICITY" using the Boolean operator OR. In the 1994 and 1999 subsets, we performed head-to-head comparisons of the currently available PubMed filter with the one we developed.

Measurements. The sensitivity, precision and the number needed to read (1/precision) were measured for different search filters.

Results. The most frequently occurring three truncated terms (diagnos*; predict* and accura*) in combination with the MeSH term "SENSITIVITY AND SPECIFICITY" produced a sensitivity of 98.1 percent (95% confidence interval: 89.9–99.9%) and a number needed to read of 8.3 (95% confidence interval: 6.7–11.3%). In direct comparisons of the new filter with the currently available one in PubMed using the 1994 and 1999 subsets, the new filter achieved better precision (12.0% versus 8.2% in 1994 and 5.0% versus 4.3% in 1999. The 95% confidence intervals for the differences range from 0.05% to 7.5% (p = 0.041) and –1.0% to 2.3% (p = 0.45), respectively). The new filter achieved slightly better sensitivities than the currently available one in both subsets, namely 98.1 and 96.1% (p = 0.32) versus 95.1 and 88.8% (p = 0.125).

Conclusions. The quoted performance of the currently available filter for diagnostic studies in PubMed may be overstated. It appears that even single external validation may lead to over optimistic views of a filter’s performance. Precision appears to be more unstable than sensitivity. In terms of sensitivity, our filter for diagnostic studies performed slightly better than the currently available one and it performed better with regards to precision in the 1994 subset. Additional research is required to determine whether these improvements are beneficial to searches in practice.

This article has been cited by other articles:

				J. S Cnossen, K. C Vollebregt, N. d. Vrieze, G. t. Riet, B. W J Mol, A. Franx, K. S Khan, and J. A M v. d. Post Accuracy of mean arterial pressure and blood pressure measurements in predicting pre-eclampsia: systematic review and meta-analysis BMJ, May 17, 2008; 336(7653): 1117 - 1120. [Abstract] [Full Text] [PDF]

				M. P. Astin, M. G. Brazzelli, C. M. Fraser, C. E. Counsell, G. Needham, and J. M. Grimshaw Developing a Sensitive Search Strategy in MEDLINE to Retrieve Studies on Assessment of the Diagnostic Performance of Imaging Techniques Radiology, May 1, 2008; 247(2): 365 - 373. [Abstract] [Full Text] [PDF]

				N. L. Wilczynski, S. Marks, and R. B. Haynes Search Strategies for Identifying Qualitative Studies in CINAHL Qual Health Res, May 1, 2007; 17(5): 705 - 710. [Abstract] [PDF]

				C. Gatsonis and P. Paliwal Meta-analysis of diagnostic and screening test accuracy evaluations: methodologic primer. Am. J. Roentgenol., August 1, 2006; 187(2): 271 - 281. [Abstract] [Full Text] [PDF]

				H. Honest, S. Sharma, and K. S. Khan Rapid Tests for Group B Streptococcus Colonization in Laboring Women: A Systematic Review Pediatrics, April 1, 2006; 117(4): 1055 - 1066. [Abstract] [Full Text] [PDF]

				E. V. Bernstam, J. R. Herskovic, Y. Aphinyanaphongs, C. F. Aliferis, M. G. Sriram, and W. R. Hersh Using Citation Data to Improve Retrieval from MEDLINE J. Am. Med. Inform. Assoc., January 1, 2006; 13(1): 96 - 105. [Abstract] [Full Text] [PDF]

				J Verbeek, J Salmi, I Pasternack, M Jauhiainen, I Laamanen, F Schaafsma, C Hulshof, and F van Dijk A search strategy for occupational health intervention studies Occup. Environ. Med., October 1, 2005; 62(10): 682 - 687. [Abstract] [Full Text] [PDF]

				N. L. Wilczynski and R. B. Haynes Optimal Search Strategies for Detecting Clinically Sound Prognostic Studies in EMBASE: An Analytic Survey J. Am. Med. Inform. Assoc., July 1, 2005; 12(4): 481 - 485. [Abstract] [Full Text] [PDF]

				J. S. Cnossen, B. W. Mol, J. A. M. van Der Post, P. M. M. Bossuyt, and G. ter Riet World Health Organization Systematic Review of Screening Tests for Preeclampsia Obstet. Gynecol., May 1, 2005; 105(5): 1151 - 1152. [Full Text] [PDF]

				Y. Aphinyanaphongs, I. Tsamardinos, A. Statnikov, D. Hardin, and C. F. Aliferis Text Categorization Models for High-Quality Article Retrieval in Internal Medicine J. Am. Med. Inform. Assoc., March 1, 2005; 12(2): 207 - 216. [Abstract] [Full Text] [PDF]

				G. G. Biondi-Zoccai, P. Agostoni, A. Abbate, L. Testa, and F. Burzotta A simple hint to improve Robinson and Dickersin's highly sensitive PubMed search strategy for controlled clinical trials Int. J. Epidemiol., February 1, 2005; 34(1): 224 - 225. [Full Text] [PDF]

				B. Smith and J. A Cleland Is radiologic examination necessary for a 9-year-old girl with a knee injury? Physical Therapy, November 1, 2004; 84(11): 1092 - 1100. [Full Text] [PDF]

				R B. Haynes and N. L Wilczynski Optimal search strategies for retrieving scientifically strong studies of diagnosis from Medline: analytical survey BMJ, May 1, 2004; 328(7447): 1040. [Abstract] [Full Text] [PDF]

				R. P Rietveld, H. C P M van Weert, G. t. Riet, and P. J E Bindels Diagnostic impact of signs and symptoms in acute infectious conjunctivitis: systematic literature search BMJ, October 4, 2003; 327(7418): 789 - 789. [Full Text] [PDF]