help button home button JAMIA Bigger figures
HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH
First published August 21, 2007 as JAMIA PrePrint; doi:10.1197/jamia.M2462
Journal of the American Medical Informatics Association 2007;14(6):746-755
© 2007 American Medical Informatics Association

A more recent version of this article appeared on November 1, 2007
This Article
Right arrow Full Text (PDF)
Right arrow All Versions of this Article:
M2462v1
14/6/746    most recent
Right arrow Submit a response
Right arrow Alert me when this article is cited
Right arrow Alert me when eLetters are posted
Right arrow Alert me if a correction is posted
Services
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Right arrow reprints & permissions
Citing Articles
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Westbrook, J. I.
Right arrow Articles by Iedema, R.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Westbrook, J. I.
Right arrow Articles by Iedema, R.

Submitted on April 2, 2007
Accepted on July 29, 2007

Multi-method evaluation of information and communication technologies in health in the context of wicked problems and socio-technical theory

Johanna I. Westbrook PhD1*, Jeffrey Braithwaite PhD2, Andrew Georgiou MSc1, Amanda Ampt MHIM1, Nerida Creswick BAppSc(HIM)1, Enrico Coiera PhD3, and Rick Iedema PhD4

Affiliation of the authors: 1 Health Informatics Research & Evaluation Unit, University of Sydney, New South Wales, Australia ; 2 Centre for Clinical Governance Research in Health, University of New South Wales, Sydney, New South Wales, Australia ; 3 Centre for Health Informatics, University of New South Wales, Sydney, New South Wales, Australia ; 4 University of Technology, Sydney, Australia

* To whom correspondence should be addressed.

Objective Few research designs look at the deep structure of complex social systems. We report the design and implementation of a multi-method evaluation model to assess the impact of computerized order entry systems on both the technical and social systems within a health care organization.

Design We designed a multi-method evaluation model informed by socio-technical theory and an appreciation of the nature of wicked problems. We mobilized this model to assess the impact of an electronic medication management system via a three year program of research at a major academic hospital.

Measurements Model components include measurements relating to three dimensions of system impact: safety and quality; organizational culture; and work and communication patterns.

Results Application of the evaluation model required the development and testing of purpose-built measurement tools such as software to collect multi-dimensional work measurement data. The model applied established research methods including medication error audits and social network analysis. Design features of these tools and techniques are described along with the practical challenges of their implementation. The distinctiveness of doing research within a unique paradigm of complex systems, explicating the wickedness and the dimensionality of socio-technical theory, is articulated.

Conclusions Designing an effective evaluation model requires a deep understanding of the nature and complexity of the problems for which information technology interventions in health care are trying to address. Adopting a socio-technical perspective for model generation improves our ability to develop evaluation models which are adaptive and sensitive to the characteristics of wicked problems and provides a strong theoretical basis from which to analyze and interpret findings.






HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH
Copyright © 1994 by the American Medical Informatics Association.