Kawamoto K et al. 2005 "Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features critical to success."
Reference
Kawamoto K, Houlihan CA, Balas EA, et al. Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features critical to success. Br Med J 2005;330(7494):765-768.
Abstract
"Objective: To identify features of clinical decision support systems critical for improving clinical practice. Design: Systematic review of randomised controlled trials. Data sources: Literature searches via Medline, CINAHL, and the Cochrane Controlled Trials Register up to 2003; and searches of reference lists of included studies and relevant reviews. Study selection: Studies had to evaluate the ability of decision support systems to improve clinical practice. Data extraction: Studies were assessed for statistically and clinically significant improvement in clinical practice and for the presence of 15 decision support system features whose importance had been repeatedly suggested in the literature. Results: Seventy studies were included. Decision support systems significantly improved clinical practice in 68% of trials. Univariate analyses revealed that, for five of the system features, interventions possessing the feature were significantly more likely to improve clinical practice than interventions lacking the feature. Multiple logistic regression analysis identified four features as independent predictors of improved clinical practice: automatic provision of decision support as part of clinician workflow (P < 0.00001), provision of recommendations rather than just assessments (P = 0.0187), provision of decision support at the time and location of decision making (P = 0.0263), and computer based decision support (P = 0.0294). Of 32 systems possessing all four features, 30 (94%) significantly improved clinical practice. Furthermore, direct experimental justification was found for providing periodic performance feedback, sharing recommendations with patients, and requesting documentation of reasons for not following recommendations. Conclusions: Several features were closely correlated with decision support systems' ability to improve patient care significantly. Clinicians and other stakeholders should implement clinical decision support systems that incorporate these features whenever feasible and appropriate."
Objective
"To identify features of clinical decision support systems critical for improving clinical practice."
Size
not applicable
Other Information
The 70 studies included approximately 6,000 clinicians and 130,000 patients.
Type of Health IT
Decision support system
Type of Health IT Functions
The authors defined a clinical decision support system (CDSS) as, "any electronic or non-electronic system designed to aid directly in clinical decision making, in which characteristics of individual patients are used to generate patient-specific assessments or recommendations that are then presented to clinicians for consideration." The 15 identified features were "integration with charting or order entry system to support workflow integration," "use of a computer to generate the decision support," "automatic provision of decision support as part of clinician workflow," "no need for additional clinician data entry," "request documentation of the reason for not following CDSS recommendations," "provision of decision support at time and location of decision making," "recommendations executed by noting agreement," "provision of a recommendation, not just an assessment," "promotion of action rather than inaction," "justification of decision support via provision of reasoning," "justification of decision support via provision of research evidence," "local user involvement in development process," "provision of decision support results to patients as well as providers," "CDSS accompanied by periodic performance feedback," and "CDSS accompanied by conventional education."
Workflow-Related Findings
"Similarly, systems that were provided as an integrated component of charting or order entry systems were significantly more likely to succeed than stand alone systems (rate difference 37% (6% to 61%))."
"Most notably, [meta-regression analysis] confirmed the critical importance of automatically providing decision support as part of the clinician workflow (P < 0.00001)."
"...systems that prompted clinicians to record a reason when not following the advised course of action were significantly more likely to succeed than systems that allowed the system advice to be bypassed without recording a reason (rate difference 41% (19% to 54%));" Other analyses confirmed this finding.
"...systems that provided a recommendation (such as "Patient is at high risk of coronary artery disease; recommend initiation of beta blocker therapy") were significantly more likely to succeed than systems that provided only an assessment of the patient (such as "Patient is at high risk of coronary artery disease") (rate difference 35% (8% to 58%))." Other analyses confirmed this finding.
"Finally, systems that provided decision support at the time and location of decision making were substantially more likely to succeed than systems that did not provide advice at the point of care, but the difference in success rates fell just short of being significant at the 0.05 level (rate difference 48% (0.46% to 70.01%))."
"In contrast, clinical decision support system effectiveness remained largely unchanged when critiques were worded more strongly..."
"Clinical decision support system effectiveness remained largely unchanged when...the evidence supporting the critiques was expanded to include institution-specific data, when recommendations were made more specific, when local clinicians were recruited into the system development process, and when bibliographic citations were provided to support the recommendations made by the system."
"A common theme among all four features [of successful systems] is that they make it easier for clinicians to use a clinical decision support system. For example, automatically providing decision support eliminates the need for clinicians to seek out the advice of the system, and the use of a computer system improves the consistency and reliability of the clinical decision support system by minimising labour intensive and error prone processes such as manual chart abstractions. As a general principle, then, our findings suggest that an effective clinical decision support system must minimise the effort required by clinicians to receive and act on system recommendations."
"When feasible and appropriate, clinical decision support systems should also provide periodic performance feedback, request documentation of the reason for not following system recommendations, and share decision support results with patients."
"Most notably, 75% of interventions succeeded when the decision support was provided to clinicians automatically, whereas none succeeded when clinicians were required to seek out the advice of the decision support system (rate difference 75% (37% to 84%))."
Study Design
Systematic literature review
Study Participants
Eighty-eight papers discussing 70 studies describing CDSSs and their ability to improve clinical practice were included. The authors included in the review "any randomised controlled trial evaluating the ability of a clinical decision support system to improve an important clinical practice in a real clinical setting."