help button home button JAMIA Hate scrolling?
HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS
This Article
Right arrow Full Text (PDF)
Right arrow Submit a response
Right arrow Alert me when this article is cited
Right arrow Alert me when eLetters are posted
Right arrow Alert me if a correction is posted
Services
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Right arrow reprints & permissions
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Detmer, W. M.
Right arrow Articles by Fagan, L. M.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Detmer, W. M.
Right arrow Articles by Fagan, L. M.

Journal of the American Medical Informatics Association, Vol 2, 46-57, Copyright © 1995 by American Medical Informatics Association

ARTICLES

A continuous-speech interface to a decision support system: II. An evaluation using a Wizard-of-Oz experimental paradigm

WM Detmer, S Shiffman, JC Wyatt, CP Friedman, CD Lane and LM Fagan
Section on Medical Informatics, Stanford University School of Medicine, CA 94305-5479.

OBJECTIVE: Evaluate the performance of a continuous-speech interface to a decision support system. DESIGN: The authors performed a prospective evaluation of a speech interface that matches unconstrained utterances of physicians with controlled-vocabulary terms from Quick Medical Reference (QMR). The performance of the speech interface was assessed in two stages: in the real-time experiment, physician subjects viewed audiovisual stimuli intended to evoke clinical findings, spoke a description of each finding into the speech interface, and then chose from a list generated by the interface the QMR term that most closely matched the finding. Subjects believed that the speech recognizer decoded their utterances; in reality, a hidden experimenter typed utterances into the interface (Wizard-of-Oz experimental design). Later, the authors replayed the same utterances through the speech recognizer and measured how accurately utterances matched with appropriate QMR terms using the results of the real-time experiment as the "gold standard." MEASUREMENTS: The authors measured how accurately the speech-recognition system converted input utterances to text strings (recognition accuracy) and how accurately the speech interface matched input utterances to appropriate QMR terms (semantic accuracy). RESULTS: Overall recognition accuracy was less than 50%. However, using language-processing techniques that match keywords in recognized utterances to keywords in QMR terms, the semantic accuracy of the system was 81%. CONCLUSIONS: Reasonable semantic accuracy was attained when language-processing techniques were used to accommodate for speech misrecognition. In addition, the Wizard-of-Oz experimental design offered many advantages for this evaluation. The authors believe that this technique may be useful to future evaluators of speech-input systems.


This article has been cited by other articles:


Home page
 J. Am. Med. Inform. Assoc.Home page
A. Zafar, J. M. Overhage, and C. J. McDonald
Continuous Speech Recognition for Clinicians
J. Am. Med. Inform. Assoc., May 1, 1999; 6(3): 195 - 204.
[Abstract] [Full Text] [PDF]

Home page
 J. Am. Med. Inform. Assoc.Home page
M. M. Kuilboer, J. van der Lei, J. C. de Jongste, S. E. Overbeek, B. Ponsioen, and J. H. van Bemmel
Simulating an Integrated Critiquing System
J. Am. Med. Inform. Assoc., March 1, 1998; 5(2): 194 - 202.
[Abstract] [Full Text]


HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS
Copyright © 1995 by the American Medical Informatics Association.