As a part of the i2b2 (Informatics for Integrating Biology to the Bedside) project, in order to survey, facilitate, and examine studies in medical language understanding for clinical narratives, the authors organized a Natural Language Processing (NLP) challenge on automatically determining the smoking status of patients from information found in their discharge records. This manuscript provides an overview of this smoking challenge, describes the data and the annotation process, explains the evaluation metrics, discusses the characteristics of the systems developed for the challenge, presents an analysis of the results of received system runs, draws conclusions about the state-of-the-art, and identifies directions for future research. A total of 11 teams participated in the smoking challenge. Each team submitted up to three system runs, providing a total of 23 submissions. The submitted system runs were evaluated with micro and macro averaged precision, recall, and F measure. The systems submitted to the smoking challenge represented a variety of machine learning and rule based algorithms. Despite the differences in their approaches to smoking status identification, many of these systems provided good results. There were twelve system runs with micro averaged F measures above 0.84. Analysis of the results highlighted the fact that discharge summaries express smoking status using a limited number of textual features (e.g., "smok", "tobac", "cigar", SOCIAL HISTORY, etc.). Many of the effective smoking status identifiers benefit from these features.