Steele AW et al. 2005 "The effect of automated alerts on provider ordering behavior in an outpatient setting."

Steele AW, Eisert S, Witter J, et al. The effect of automated alerts on provider ordering behavior in an outpatient setting. PLoS Med 2005;2(9):864-870.
"Background: Computerized order entry systems have the potential to prevent medication errors and decrease adverse drug events with the use of clinical-decision support systems presenting alerts to providers. Despite the large volume of medications prescribed in the outpatient setting, few studies have assessed the impact of automated alerts on medication errors related to drug-laboratory interactions in an outpatient primary-care setting.
Methods and Findings: A primary-care clinic in an integrated safety net institution was the setting for the study. In collaboration with commercial information technology vendors, rules were developed to address a set of drug-laboratory interactions. All patients seen in the clinic during the study period were eligible for the intervention. As providers ordered medications on a computer, an alert was displayed if a relevant drug-laboratory interaction existed. Comparisons were made between baseline and postintervention time periods. Provider ordering behavior was monitored focusing on the number of medication orders not completed and the number of rule-associated laboratory test orders initiated after alert display. Adverse drug events were assessed by doing a random sample of chart reviews using the Naranjo scoring scale. The rule processed 16,291 times during the study period on all possible medication orders: 7,017 during the pre-intervention period and 9,274 during the postintervention period. During the postintervention period, an alert was displayed for 11.8% (1,093 out of 9,274) of the times the rule processed, with 5.6% for only "missing laboratory values," 6.0% for only "abnormal laboratory values," and 0.2% for both types of alerts. Focusing on 18 high-volume and high-risk medications revealed a significant increase in the percentage of time the provider stopped the ordering process and did not complete the medication order when an alert for an abnormal rule-associated laboratory result was displayed (5.6% vs. 10.9%, p=0.03, Generalized Estimating Equations test). The provider also increased ordering of the rule-associated laboratory test when an alert was displayed (39% at baseline vs. 51% during post intervention, p = 0.001). There was a non-statistically significant difference towards less "definite" or "probable" adverse drug events defined by Naranjo scoring (10.3% at baseline vs. 4.3% during postintervention, p = 0.23).
Conclusion: Providers will adhere to alerts and will use this information to improve patient care. Specifically, in response to drug-laboratory interaction alerts, providers will significantly increase the ordering of appropriate laboratory tests. There may be a concomitant change in adverse drug events that would require a larger study to confirm. Implementation of rules technology to prevent medication errors could be an effective tool for reducing medication errors in an outpatient setting."
To assess "the impact of automated alerts on medication errors related to [missing or abnormal lab values] in an outpatient primary-care setting."
Type Clinic
Primary care
Other Information
The study took place at a large outpatient clinic of Denver Health, the primary safety net institution in Denver, Colorado.
Type of Health IT
Computerized clinical reminders (CRs) and alerts
Type of Health IT Functions
"If criteria were met [indicating the possibility of an adverse drug event], an alert screen was presented to the provider with a message containing patient name, type of [criteria] alert, name of medication that triggered the alert, and a message with laboratory results, if available, and suggestions to consider deleting or changing the medication or to consider ordering [an] associated laboratory test. Providers did not need to respond to the alert, but needed to select "Continue" to proceed with the ordering session."
Context or other IT in place
An electronic health record (EHR) application, computerized provider order entry (CPOE) system, and computerized decision support system (CDSS) were all in place.
Workflow-Related Findings
Between pre- and post-intervention study periods, there was not a statistical difference in the rate of providers not completing medication orders when encountering any type of alert (5.4 percent pre-intervention vs. 8.3 percent post-intervention. However, when providers encountered an alert for an abnormal lab value, the percentage of medication orders not completed increased (5.6 percent pre-intervention vs. 10.9 percent post-intervention).
"On average, the rule delayed processing of the screens in the CPOE application by less than 2 sec[onds]."
"There were no complaints from providers about slow system performance related to the rules processing during this study."
For medication orders triggering an alert, the percentage of time the provider ordered the suggested laboratory test increased from 38.5 percent to 51.1 percent. The effect was largest effect when a laboratory test was missing; the percentage of times the test was ordered increased from 43.0 percent at baseline to 62.0 percent.
Study Design
Pre-postintervention (no control group)
Study Participants
"All provider staff were allowed to enter medication orders including physicians, allied health providers (nurse practitioners, physician assistants), and residents." Alerts would fire for any patient that was being prescribed a medication that, in combination with missing or abnormal lab values, could cause one of several specific types of adverse drug events.