This Article Full Text Full Text (PDF) All Versions of this Article: M1794v1 12/4/448    most recent Submit a response Alert me when this article is cited Alert me when eLetters are posted Alert me if a correction is posted Services Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Melton, G. B. Articles by Hripcsak, G. Search for Related Content PubMed PubMed Citation Articles by Melton, G. B. Articles by Hripcsak, G.

Automated Detection of Adverse Events Using Natural Language Processing of Discharge Summaries

Affiliations of the authors: Department of Biomedical Informatics, Columbia University (GBM, GH); and Medical Informatics Services, NewYork-Presbyterian Hospital (GH), New York, NY.

Correspondence and reprints: George Hripcsak, MD, MS, Department of Biomedical Informatics, Columbia University, 622 West 168th Street, Vanderbilt Clinic, 5th Floor, New York, NY 10032; e-mail: <hripcsak{at}columbia.edu>.

Received for publication: 01/19/05; accepted for publication: 03/20/05.

Objective: To determine whether natural language processing (NLP) can effectively detect adverse events defined in the New York Patient Occurrence Reporting and Tracking System (NYPORTS) using discharge summaries.

Design: An adverse event detection system for discharge summaries using the NLP system MedLEE was constructed to identify 45 NYPORTS event types. The system was first applied to a random sample of 1,000 manually reviewed charts. The system then processed all inpatient cases with electronic discharge summaries for two years. All system-identified events were reviewed, and performance was compared with traditional reporting.

Measurements: System sensitivity, specificity, and predictive value, with manual review serving as the gold standard.

Results: The system correctly identified 16 of 65 events in 1,000 charts. Of 57,452 total electronic discharge summaries, the system identified 1,590 events in 1,461 cases, and manual review verified 704 events in 652 cases, resulting in an overall sensitivity of 0.28 (95% confidence interval [CI]: 0.17–0.42), specificity of 0.985 (CI: 0.984–0.986), and positive predictive value of 0.45 (CI: 0.42–0.47) for detecting cases with events and an average specificity of 0.9996 (CI: 0.9996–0.9997) per event type. Traditional event reporting detected 322 events during the period (sensitivity 0.09), of which the system identified 110 as well as 594 additional events missed by traditional methods.

Conclusion: NLP is an effective technique for detecting a broad range of adverse events in text documents and outperformed traditional and previous automated adverse event detection methods.

This article has been cited by other articles:

				P. M. Kilbridge and D. C. Classen The Informatics Opportunities at the Intersection of Patient Safety and Clinical Informatics J. Am. Med. Inform. Assoc., July 1, 2008; 15(4): 397 - 407. [Abstract] [Full Text] [PDF]

				S. Pakhomov, S. Bjornsen, P. Hanson, and S. Smith Quality Performance Measurement Using the Text of Electronic Medical Records Med Decis Making, July 1, 2008; 28(4): 462 - 470. [Abstract] [PDF]

				S. V.S. Pakhomov, P. L. Hanson, S. S. Bjornsen, and S. A. Smith Automatic Classification of Foot Examination Findings Using Clinical Notes and Machine Learning J. Am. Med. Inform. Assoc., March 1, 2008; 15(2): 198 - 202. [Abstract] [Full Text] [PDF]

				O. Uzuner, I. Goldstein, Y. Luo, and I. Kohane Identifying Patient Smoking Status from Medical Discharge Records J. Am. Med. Inform. Assoc., January 1, 2008; 15(1): 14 - 24. [Abstract] [Full Text] [PDF]

				V. L. Hinrichsen, B. Kruskal, M. A. O'Brien, T. A. Lieu, R. Platt, and Vaccine Safety Datalink Team Using Electronic Medical Records to Enhance Detection and Reporting of Vaccine Adverse Events J. Am. Med. Inform. Assoc., November 1, 2007; 14(6): 731 - 735. [Abstract] [Full Text] [PDF]

				A. Wright, H. Goldberg, T. Hongsermeier, and B. Middleton A Description and Functional Taxonomy of Rule-based Decision Support Content at a Large Integrated Delivery Network J. Am. Med. Inform. Assoc., July 1, 2007; 14(4): 489 - 496. [Abstract] [Full Text] [PDF]

				M. N Cantor, H. J Feldman, and M. M Triola Using trigger phrases to detect adverse drug reactions in ambulatory care notes Qual. Saf. Health Care, April 1, 2007; 16(2): 132 - 134. [Abstract] [Full Text] [PDF]

				A. Turchin, N. S. Kolatkar, R. W. Grant, E. C. Makhni, M. L. Pendergrass, and J. S. Einbinder Using Regular Expressions to Abstract Blood Pressure and Treatment Intensification Information from the Text of Physician Notes J. Am. Med. Inform. Assoc., November 1, 2006; 13(6): 691 - 695. [Abstract] [Full Text] [PDF]

				E. C. Wu and N. Shah Corporate strategies for computerization. Ann Intern Med, September 5, 2006; 145(5): 395 - 395. [Full Text] [PDF]

				P M Kilbridge and D C Classen Automated surveillance for adverse events in hospitalized patients: back to the future. Qual. Saf. Health Care, June 1, 2006; 15(3): 148 - 149. [Full Text] [PDF]

				M K Szekendi, C Sullivan, A Bobb, J Feinglass, D Rooney, C Barnard, and G A Noskin Active surveillance using electronic triggers to detect adverse events in hospitalized patients. Qual. Saf. Health Care, June 1, 2006; 15(3): 184 - 190. [Abstract] [Full Text] [PDF]

				R. A. Miller, R. M. Gardner, K. B. Johnson, and G. Hripcsak Clinical Decision Support and Electronic Prescribing Systems: A Time for Responsible Thought and Action J. Am. Med. Inform. Assoc., July 1, 2005; 12(4): 403 - 409. [Full Text] [PDF]