Shojania K et al. 2009 "The effects of on-screen, point of care computer reminders on processes and outcomes of care."
Reference
Shojania K, Jennings, A, Mayhew, A, Ramsay, CR, Eccles, MP, and Grimshaw, J. The effects of on-screen, point of care computer reminders on processes and outcomes of care. Cochrane Database Syst Rev 2009;3.
Abstract
"Background: The opportunity to improve care by delivering decision support to clinicians at the point of care represents one of the main incentives for implementing sophisticated clinical information systems. Previous reviews of computer reminder and decision support systems have reported mixed effects, possibly because they did not distinguish point of care computer reminders from e-mail alerts, computer-generated paper reminders, and other modes of delivering 'computer reminders'.
Objectives: To evaluate the effects on processes and outcomes of care attributable to on-screen computer reminders delivered to clinicians at the point of care.
Search strategy: We searched the Cochrane EPOC Group Trials register, MEDLINE, EMBASE and CINAHL and CENTRAL to July 2008, and scanned bibliographies from key articles.
Selection criteria: Studies of a reminder delivered via a computer system routinely used by clinicians, with a randomised or quasi-randomised design and reporting at least one outcome involving a clinical endpoint or adherence to a recommended process of care.
Data collection and analysis: Two authors independently screened studies for eligibility and abstracted data. For each study, we calculated the median improvement in adherence to target processes of care and also identified the outcome with the largest such improvement. We then calculated the median absolute improvement in process adherence across all studies using both the median outcome from each study and the best outcome.
Main results: Twenty-eight studies (reporting a total of thirty-two comparisons) were included. Computer reminders achieved a median improvement in process adherence of 4.2% (interquartile range (IQR): 0.8% to 18.8%) across all reported process outcomes, 3.3% (IQR: 0.5% to 10.6%) for medication ordering, 3.8% (IQR: 0.5% to 6.6%) for vaccinations, and 3.8% (IQR: 0.4% to 16.3%) for test ordering. In a sensitivity analysis using the best outcome from each study, the median improvement was 5.6% (IQR: 2.0% to 19.2%) across all process measures and 6.2% (IQR: 3.0% to 28.0%) across measures of medication ordering. In the eight comparisons that reported dichotomous clinical endpoints, intervention patients experienced a median absolute improvement of 2.5% (IQR: 1.3% to 4.2%). Blood pressure was the most commonly reported clinical endpoint, with intervention patients experiencing a median reduction in their systolic blood pressure of 1.0 mmHg (IQR: 2.3 mmHg reduction to 2.0 mmHg increase).
Authors conclusions: Point of care computer reminders generally achieve small to modest improvements in provider behaviour. A minority of interventions showed larger effects, but no specific reminder or contextual features were significantly associated with effect magnitude. Further research must identify design features and contextual factors consistently associated with larger improvements in provider behaviour if computer reminders are to succeed on more than a trial and error basis."
Objectives: To evaluate the effects on processes and outcomes of care attributable to on-screen computer reminders delivered to clinicians at the point of care.
Search strategy: We searched the Cochrane EPOC Group Trials register, MEDLINE, EMBASE and CINAHL and CENTRAL to July 2008, and scanned bibliographies from key articles.
Selection criteria: Studies of a reminder delivered via a computer system routinely used by clinicians, with a randomised or quasi-randomised design and reporting at least one outcome involving a clinical endpoint or adherence to a recommended process of care.
Data collection and analysis: Two authors independently screened studies for eligibility and abstracted data. For each study, we calculated the median improvement in adherence to target processes of care and also identified the outcome with the largest such improvement. We then calculated the median absolute improvement in process adherence across all studies using both the median outcome from each study and the best outcome.
Main results: Twenty-eight studies (reporting a total of thirty-two comparisons) were included. Computer reminders achieved a median improvement in process adherence of 4.2% (interquartile range (IQR): 0.8% to 18.8%) across all reported process outcomes, 3.3% (IQR: 0.5% to 10.6%) for medication ordering, 3.8% (IQR: 0.5% to 6.6%) for vaccinations, and 3.8% (IQR: 0.4% to 16.3%) for test ordering. In a sensitivity analysis using the best outcome from each study, the median improvement was 5.6% (IQR: 2.0% to 19.2%) across all process measures and 6.2% (IQR: 3.0% to 28.0%) across measures of medication ordering. In the eight comparisons that reported dichotomous clinical endpoints, intervention patients experienced a median absolute improvement of 2.5% (IQR: 1.3% to 4.2%). Blood pressure was the most commonly reported clinical endpoint, with intervention patients experiencing a median reduction in their systolic blood pressure of 1.0 mmHg (IQR: 2.3 mmHg reduction to 2.0 mmHg increase).
Authors conclusions: Point of care computer reminders generally achieve small to modest improvements in provider behaviour. A minority of interventions showed larger effects, but no specific reminder or contextual features were significantly associated with effect magnitude. Further research must identify design features and contextual factors consistently associated with larger improvements in provider behaviour if computer reminders are to succeed on more than a trial and error basis."
Objective
"To evaluate the effects on processes and outcomes of care attributable to on-screen computer reminders delivered to clinicians at the point of care." The authors addressed, "...the following questions: 1. Do on-screen computer reminders effectively improve processes or outcomes of care? 2. Do any readily identifiable elements of on-screen reminders influence their effectiveness (e.g. inclusion of patient-specific information as opposed to generic reminders for a given condition, requiring a response from users). 3. Do any readily identifiable elements of the targeted activity (e.g. chart documentation, test ordering, medication prescribing) influence the effectiveness of on-screen reminders?"
Size
not applicable
Type of Health IT
Decision support system
Type of Health IT Functions
"The original protocol for this review defined 'on-screen computer reminders' as follows: Patient or encounter specific information that is provided via a computer
console (either visually or audibly) and intended to prompt a healthcare professional to recall information usually encountered through their general medical education, in the medical records or through interaction with peers, and so remind them to perform or avoid some action to aid individual patient care...By 'point of care' we refer to delivery of the computer reminder to clinicians at the time they are engaged in the target activity of interest, such as prescribing medications, documenting clinical encounters in the medical record, and ordering investigations."
console (either visually or audibly) and intended to prompt a healthcare professional to recall information usually encountered through their general medical education, in the medical records or through interaction with peers, and so remind them to perform or avoid some action to aid individual patient care...By 'point of care' we refer to delivery of the computer reminder to clinicians at the time they are engaged in the target activity of interest, such as prescribing medications, documenting clinical encounters in the medical record, and ordering investigations."
Workflow-Related Findings
"...the median improvements in process adherence associated with computer reminders were: 4.2 % (IQR: 0.8% to 18.8%) across all process outcomes, 3.3% (IQR: 0.5% to 10.6%) for improvements in prescribing behaviours, 3.8% (IQR: 0.5% to 6.6%) for improvements in vaccination, and 3.8%(IQR: 0.4% to 16.3%) for test ordering behaviours."
"The degree of improvement did not differ significantly between studies based on the type of quality problem targeted (underuse versus overuse of a given process of care), the conveyance of patient specific information versus a more generic alert, provision of an explanation for the alert, whether or not the reminder conveyed a specific recommendation, whether or not the authors of the study had developed the reminder, or the type of system used to deliver the reminder (CPOE versus electronic medical record)."
"There was a trend towards larger effects with reminders that required users to enter a response of some kind (12.9%, IQR 2.7% to 22.7%) versus those that did not (2.7%, IQR: 0.6% to 5.6%; P = 0.09)."
"We also analysed whether effect sizes differed between reminders that were 'pushed' onto users (that is users automatically received the reminder) versus reminders that required users to perform some action to receive it (that is users had to 'pull' the reminders). Only four comparisons involved 'pull' reminders and these showed comparable effects to 'push' reminders."
Study Design
Systematic literature review
Study Participants
The authors "identified 2036 citations, of which 1662 were excluded at the initial stage of screening and an additional 374 on full-text review, yielding a total of 28 articles that met all inclusion criteria ... resulting in 32 comparisons." They "included randomised controlled trials (with randomisation at the level of the patient or the provider) and quasi-randomised trials, where allocation to intervention or control occurred on the basis of an arbitrary but not truly random process (for example even or odd patient identification numbers)... Any study in which the majority of participants (> 50%) consisted of physicians or physician trainees [were also included]." The authors "excluded studies that primarily targeted dentists, pharmacists, nurses, or other health professionals."