Objective We developed and evaluated a rating scale, the Attitudes Toward Handheld Decision Support Software Scale (H-DSS), to assess physician attitudes about handheld decision support systems.

Design We conducted a prospective assessment of psychometric characteristics of the H-DSS including reliability, validity, and responsiveness. Participants were 82 Internal Medicine residents. A higher score on each of the 14 five-point Likert scale items reflected a more positive attitude about handheld DSS. The H-DSS score is the mean across the fourteen items. Attitudes toward the use of the handheld DSS were assessed prior to and six months after receiving the handheld device.

Statistics Cronbach's Alpha was used to assess internal consistency reliability. Pearson correlations were used to estimate and detect significant associations between scale scores and other measures (validity). Paired sample t-tests were used to test for changes in the mean attitude scale score (responsiveness) and for differences between groups.

Results Internal consistency reliability for the scale was a = 0.73. In testing validity, moderate correlations were noted between the attitude scale scores and self-reported PDA usage in the hospital (correlation coefficient = 0.55) and clinic (0.48), p < 0.05 for both. The scale was responsive, in that it detected the expected increase in scores between the two administrations (3.99 (s.d. = 0.35) vs. 4.08, (s.d. = 0.34), p < 0.005).

Conclusion Our evaluation showed that the H-DSS scale was reliable, valid and responsive. The scale can be used to guide future handheld DSS development and implementation.