Objective Adverse events are poor health outcomes caused by medical care. Measuring them is necessary for quality improvements but current detection methods are inadequate. We performed this study to validate a previously derived method of adverse event detection using term searching in physician dictated discharge summaries.

Design Retrospective, chart review study.

Population A random sample of 245 adult medicine and surgery patients admitted to a multi-campus academic medical center in 2002.

Measurements We used a commercially available search engine to scan discharge summaries for the presence of 104 terms that potentially indicate an adverse event. Summaries with any of these terms were reviewed by a physician to determine the term's context. Screen-positive summaries had a term that was contextually indicative of an adverse event. We used a two-staged chart review as the gold standard to determine the true presence or absence of an adverse event.

Results The average patient age was 62 years (SD 18.6) and 55% were admitted to a medical service. By gold standard criteria, 48 of 245 patients had an adverse event. Term searching classified 27 cases with an adverse event, with 11 true positives; 218 cases were classified as not having an adverse event, with 181 true negatives. The sensitivity, specificity, and positive and negative predictive values were, 0.23 (95% CI 0.11-0.35), 0.92 (0.88-0.96), 0.41 (0.25-0.59), and 0.83 (95% 0.77-0.97) respectively.

Discussion Although the sensitivity of the method is low, its high specificity means that the method could be used to replace expensive manual chart reviews by nurses.