Shiffman RN et al. 1999 "Computer-based guideline implementation systems: a systematic review of functionality and effectiveness."

Shiffman RN, Liaw Y, Brandt CA, et al. Computer-based guideline implementation systems: a systematic review of functionality and effectiveness. J Am Med Inform Assoc 1999;6(2):104-114.
"In this systematic review, the authors analyze the functionality provided by recent computer-based guideline implementation systems and characterize the effectiveness of the systems. Twenty-five studies published between 1992 and January 1998 were identified. Articles were included if the authors indicated an intent to implement guideline recommendations for clinicians and if the effectiveness of the system was evaluated. Provision of eight information management services and effects on guideline adherence, documentation, user satisfaction, and patient outcome were noted. All systems provided patient-specific recommendations. In 19, recommendations were available concurrently with care. Explanation services were described for nine systems. Nine systems allowed interactive documentation, and 17 produced paper-based output. Communication services were present most often in systems integrated with electronic medical records. Registration, calculation, and aggregation services were infrequently reported. There were 10 controlled trials (9 randomized) and 10 time-series correlational studies. Guideline adherence improved in 14 of 18 systems in which it was measured. Documentation improved in 4 of 4 studies."
To "perform a detailed analysis of the functionality delivered by current computer-based guideline implementation systems." Also the authors sought to "analyze which information management services have been delivered by recently described guideline implementations" and to "review the effectiveness of the computer-based interventions in influencing clinicians' behavior and changing patient outcomes."
not applicable
Type of Health IT
Decision support system
Type of Health IT Functions
The authors focused on studies in which "computers were used as part of an implementation strategy for clinical practice guidelines ... not simply to provide computer-based decision support." Functions varied across systems but included the following: patient-specific recommendations; explanation functionality including rationale for recommendations and literature citations; prompts for documentation of relevant findings; reminders and alerts; patient summaries; order-entry interfaces; lab results reporting; scheduling and pharmacy interfaces; patient age calculations; preventive services (rules-based); and data capture for lab tests, demographics, admission and discharge dates and response to alerts, presented in an aggregate report.
Workflow-Related Findings
"In both studies with negative evaluations of user satisfaction, arduous data entry was suggested as a reason for poor system acceptance."
"Eighteen of the 20 studies evaluated provider adherence to the guidelines. In 14 of the 18, some level of improved adherence was described."
"User satisfaction may be affected adversely by tedious data entry requirements in the absence of offsetting system benefits."
"Four studies looked at documentation and found improvement in each case. The average number of relevant data items for surgical pre-authorization increased from 4.0 to 28.8; the mean percentage increase for documentation of common pediatric problems was 58 percent, for management of back pain 30.2 percent, and for management of exposure to body fluids 42 percent."
Study Design
Systematic literature review
Study Participants
The MEDLINE and CINAHL databases were searched from 1992 - January 1998 using the OVID search engine. The search yielded 25 papers describing 20 discrete systems.