Develop and Validate Health IT Safety Measures To Capture Violations of the Five Rights of Medication Safety

Project Final Report (PDF, 706.51 KB) Disclaimer

New measures to identify near-miss medication errors are a major advancement in patient safety and can help healthcare systems make ordering even safer. 

Project Details -
Completed

Medication errors are the most common and preventable cause of patient harm. Yet efforts to prevent these errors have been hampered by lack of standardized measures. Near-miss medication errors, such as when clinicians realize they have ordered the wrong dose or frequency for a drug, are caught before the error ever reaches the patient. Although patient harm may have been avoided, understanding the circumstances surrounding near-miss errors presents opportunities to improve safe ordering practices. Research on electronic ordering systems and processes can provide insight into contributors to medication errors, as well as insight into potential solutions.

In previous research, the team developed and validated the automated Wrong-Patient Retract-and-Reorder (RAR) Measure to identify wrong-patient electronic orders. The Wrong-Patient RAR measure identifies orders placed for a patient that are retracted within 10 minutes, and then placed by the same clinician for a different patient within the next 10 minutes, allowing for the systematic and objective identification of wrong-patient orders in electronic health record (EHR) data.

This current research expanded RAR measures needed for identifying violations of the “Five Rights of Medication Safety”: right patient, right dose, right medication, right route, and right frequency. By using the RAR methodology, the team developed new measures to capture instances where an order was placed, retracted by the ordering clinician, and subsequently reordered by the same clinician for the same patient with a parameter of the order changed.

The specific aims of the research were as follows: 

  • Develop and pilot effective and valid measures (positive predictive value >75%) for detecting wrong-dose, wrong-medication, wrong-route, and wrong-frequency electronic orders in an acute care setting, by extending the wrong-patient RAR automated detection method.
  • Implement the automated measures developed in the first aim, using a different EHR, to evaluate the reliability (external validity) of the measures.
  • Conduct a multisite observational study describing the overall frequency of wrong-patient, wrong-dose, wrong-medication, wrong-route, and wrong-frequency electronic orders, and describe the frequency in subgroups characterized by clinician, patient, and system factors.

The research team conducted a prospective study in a convenience sample of clinicians who triggered the RAR measures in development. First, the measures were developed and validated in one EHR platform by using an iterative process to improve the performance of each measure. When the target positive predictive value (PPV) was reached, the new measures were validated in a second EHR platform to demonstrate reliability. The team validated the measures by calculating their PPV, which tests how well the measures detected true errors. All measures achieved a high PPV, reaching the target of greater than 75 percent. The research showed that automated measurement of electronic order errors can be readily integrated into health system EHRs to study the epidemiology of order errors and to test the effectiveness of proposed EHR improvements on order error outcomes.

Automated Retract-and-Reorder Measures to Improve Medication Safety

Theme:

Supporting Health Systems in Advancing Care Delivery

Subtheme:

Using Digital Healthcare Tools to Improve Patient Safety

New measures to identify near-miss medication errors are a major advancement in patient safety and can help healthcare systems make ordering even safer.

Near-miss medication errors are difficult to identify and are underreported but can be opportunities to make electronic ordering safer

Medication errors are the most common and preventable cause of patient harm. Yet efforts to prevent these errors have been hampered by lack of standardized measures. Near-miss medication errors, such as when clinicians realize they’ve ordered the wrong dose or frequency for a drug, are caught before the error ever reaches the patient. Although patient harm may have been avoided, understanding the circumstances surrounding near-miss errors presents opportunities to improve safe ordering practices. Research on electronic ordering systems and processes can provide insight into contributors to medication errors, as well as insight into potential solutions.

In previous research, Dr. Jason Adelman of Columbia University developed and validated the automated Wrong-Patient Retract-and-Reorder (RAR) Measure to identify wrong-patient electronic orders. The Wrong-Patient RAR measure identifies orders placed for a patient that are retracted within 10 minutes, and then placed by the same clinician for a different patient within the next 10 minutes. These near-miss errors are self-caught by the clinician before they reach the patient. The Wrong-Patient RAR measure enabled systematic and objective identification of wrong-patient orders in electronic health record (EHR) data, resulting in a critical breakthrough in patient safety research. As Dr. Adelman noted, “Before the development of the Wrong-Patient RAR measure, there were [approximately] six voluntary reported wrong-patient order errors each year at NewYork-Presbyterian versus 10,000 errors identified in a year after the Wrong-Patient RAR measure was implemented. You can't study errors and improve systems when there are only six events.”

Expanding the RAR methodology to other medication order errors further improves understanding and safety

Based on this pioneering work, Dr. Adelman wanted to expand RAR measures to medication and other types of order errors. Using the RAR methodology, Dr. Adelman and his team developed new measures to capture instances where an order was placed, retracted by the ordering clinician, and subsequently reordered by the same clinician for the same patient with a parameter of the order changed. These measures identify electronic order errors including wrong dosage, wrong route, wrong frequency, or wrong medication.

A unique aspect of the research is that the team was able to query for RAR events every 30 minutes and then contact clinicians within 6 hours of each RAR event to understand what happened, including why the initial order was placed, why the order was canceled, who prompted the cancelation, and why the medication was reordered with a parameter changed (e.g., dose, frequency, route). This allowed the researchers to classify whether the event was a true error or not, and to better understand contributing factors for these different errors.

“This research allows us to examine the epidemiology of these errors and identify targets for intervention. Are there differences in frequency by type of errors, for example, wrong-patient versus wrong-dose errors? Are there differences by shift, for example, when you’re tired at night versus during the day? Are there differences between attendings and house staff? You can get very accurate data without the biases of chart review or voluntary reporting.”- Dr. Jason Adelman

RAR measures help identify errors and test interventions to prevent them

The RAR measures were studied at seven hospitals with over 3,000 inpatient beds and six emergency departments with two different EHR systems. The team validated the measures by calculating their positive predictive value (PPV), which tests how well the measures detected true errors. All measures achieved a high PPV, reaching the target of greater than 75 percent. The research showed that automated measurement of electronic order errors can be readily integrated into health system EHRs to study the epidemiology of order errors and to test the effectiveness of proposed EHR improvements on order error outcomes. Dr. Adelman and his team will publicly share the specifications for the RAR measures developed in this study so that the measures can be readily and widely used by other healthcare systems.

This Research is Featured in

Principal Investigator

Adelman, Jason Stuart

Organization

Columbia University Health Sciences

Research Profile

Funded Amount

$999,997

Research Type

Completed

This Research is Featured in

Related Stories




Leveraging Health IT to Test Solutions That Are Replicable, Scalable, and Improve Patient Safety

Impact

“Studies like that by Adelman and colleagues point the way to the creation of a digital learning health care system, in which the results of the interactions between clinicians (and, increasingly, patients and families) and the EHR are analyzed to help guide the strategies that lead to the highest value and most satisfying care. Having spent tens of billions of dollars digitating the health care system, it is essential to take advantage of the unique capacity of digital tools to allow clinicians and health care systems to learn from every click.” -editorial by Drs. Wachter, Murray, and Adler-Milstein4

Wrong-patient errors can affect any patient in any healthcare setting for a variety of reasons.

Dr. Jason Adelman, named as one of 50 experts leading the field of patient safety in 2018 by Becker’s Hospital Review,2 has led multiple AHRQ Health IT-funded research efforts centered on health IT safety. The research of Dr. Adelman, Executive Director of Patient Safety Research at Columbia University Irving Medical Center/NewYork-Presbyterian Hospital, is widespread and being replicated by other organizations. Dr. Adelman is part of the ECRI Patient Safety Collaborative, where he is advising several hospitals on how to implement the Wrong-Patient Retract-and-Reorder (RAR) Measure that he developed to evaluate the frequency of wrong-patient errors that occur through CPOE systems. When a clinician places an order, then cancels the order and places the same order for a different patient within the next 10 minutes, the measure flags it as a wrong-patient RAR event. While capturing the instance of retracting and reordering mistakes does not correct or prevent errors, it provides the facility with information to discover error trends along with opportunities to intercept processes that lead to such errors. Using such data as evidence of the CPOE’s potential impact on patient safety, decision makers can factor RAR measures in choosing one health IT design over another.

Having multiple EHRs open simultaneously does not increase wrong-patient orders.

“Placing orders on the wrong patient should never happen. Yet, human error is very common in the healthcare environment. Healthcare is inherently complex and heavily reliant on people rather than technology to protect patients from harm. It will likely take a multi-pronged health IT approach to prevent these types of errors.”
- Dr. Jason Adelman

Dr. Adelman’s AHRQ Health IT-funded research found that restricting clinicians to having one EHR record open at a time did not significantly reduce the rate of wrong-patient order errors compared with allowing up to four records to be open concurrently. His work did not support the ONC and Joint Commission recommendation that EHR systems should only allow one record to be displayed at a time. In the study, published in the May 14, 2019 issue of JAMA,3 using the RAR measure, Dr. Adelman and his team compared the risk of wrong-patient orders while accessing one versus four records open in a variety of clinical settings, including hospitals, EDs, and outpatient facilities. While no differences in wrong-patient orders were observed between those clinician groups, there was considerable variation in the frequency of errors in different clinical settings. The rate of wrong-patient order errors was lowest in outpatient settings, where physicians may care for one patient at a time. The highest rates, meanwhile, were seen in inpatient critical care and obstetrics units, which reflected differences in workflows and number of patients being cared for simultaneously, researchers noted. The research offers insights for healthcare systems that are trying to balance patient safety with the needs of busy clinicians who need tools for efficient workflow.

The right patient, right dose, right medication, right route, and right frequency.

The Health IT Program has also funded Dr. Adelman to extend research on RAR, to validate and evaluate the reliability of RAR using a different EHR system, so it can be implemented and may advance interventions to prevent these serious and complicated safety issues. The intent is to develop health IT measures that identify deviations from the Five Rights of Medication Safety: right patient, right dose, right medication, right route, and right frequency.

Use of photographs to prevent errors.

An additional Adelman-led, AHRQ-funded study is assessing the effectiveness of using patient photographs as an additional identifier in the EHR system to avoid wrong-patient errors when using CPOE systems. The research team will conduct a randomized controlled trial of wrong-patient error rates between systems with patient photos and without. Ultimately, the team plans to develop a toolkit with guidance to help other health systems in implementing patient photos in EHR systems.

2 https://www.beckershospitalreview.com/lists/50-experts-leading-the-field-of-patient-safety-2018.html

3 https://jamanetwork.com/journals/jama/fullarticle/2733207

4 https://jamanetwork.com/journals/jama/article-abstract/2733189