Kaplan B 2001 "Evaluating informatics applications - clinical decision support systems literature review."

Kaplan B. Evaluating informatics applications - clinical decision support systems literature review. Int J Med Inf 2001;64(1):15-37.
"This paper reviews clinical decision support systems (CDSS) literature, with a focus on evaluation. The literature indicates a general consensus that clinical decision support systems are thought to have the potential to improve care. Evidence is more equivocal for guidelines and for systems to aid physicians with diagnosis. There also is general consensus that a variety of systems are little used despite demonstrated or potential benefits. In the evaluation literature, the main emphasis is on how clinical performance changes. Most studies use an experimental or randomized controlled clinical trials design (RCT) to assess system performance or to focus on changes in clinical performance that could affect patient care. Few studies involve field tests of a CDSS and almost none use a naturalistic design in routine clinical settings with real patients. In addition, there is little theoretical discussion, although papers are permeated by a rationalist perspective that excludes contextual issues related to how and why systems are used. The studies mostly concern physicians rather than other clinicians. Further, CDSS evaluation studies appear to be insulated from evaluations of other informatics applications: Consequently, there is a lack of information useful for understanding why CDSSs may or may not be effective, resulting in making less informed decisions about these technologies and, by extension, other medical informatics applications."
To review "literature that focuses on evaluation of clinical decision support systems (CDSS)."
not applicable
Type of Health IT
Decision support system
Type of Health IT Functions
"This search identified papers classified as [being] about a (1) decision support system; (2) clinical decision support system; (3) expert system; and (4) decision aid. 'CDSS' has a variety of definitions. Any system that was considered a CDSS by the authors and catalogers of the papers reviewed was considered so for purposes of this review. Most studies use an experimental or RCT design. With only six multi-methods studies, plus three more using qualitative methods, methodological diversity is limited."
Workflow-Related Findings
"There have been substantial rates of physician noncompliance with standards. There is little evidence that physicians comply with guidelines, whether or not incorporated into a CDSS."
"Reminders, alerts, treatment plans, and patient education, have been reported as effective in changing practice behaviors. Evidence of positive effect is more equivocal for guidelines. Some studies suggest that guidelines are effective, and others that they are not ... Whether systems aid physicians with diagnosis also is unclear."
Study Design
Systematic literature review
Study Participants
Thirty-five papers were included, discussing a total of 27 studies.