Context is Critical: Understanding When and Why Electronic Health Record-Related Safety Hazards Happen (Maryland)

Project Final Report (PDF, 492.54 KB) Disclaimer

This project does not have any related annual summary.

Blackbox: A Multidisciplinary Team-Based Framework for HIT Error Pattern Detection - Final Report

Citation:
Hettinger A. Blackbox: A Multidisciplinary Team-Based Framework for HIT Error Pattern Detection - Final Report. (Prepared by MedStar Health Research Institute under Grant No. R21 HS024755). Rockville, MD: Agency for Healthcare Research and Quality, 2019. (PDF, 492.54 KB)

The findings and conclusions in this document are those of the author(s), who are responsible for its content, and do not necessarily represent the views of AHRQ. No statement in this report should be construed as an official position of AHRQ or of the U.S. Department of Health and Human Services.
Principal Investigator: 
Document Type: 
Research Method: 
This project does not have any related event.
This project does not have any related resource.
This project does not have any related survey.

Using Aviation Technology to Prevent Healthcare Errors: The Health IT Black Box

COMPLETED RESEARCH - KEY FINDINGS AND IMPACT

Similar to the airline industry’s use of a “black box” that captures actions leading up to a near miss or error, the health IT black box captures mouse movements and keystrokes made by users of EHRs. This allows for a robust analysis of the root cause of EHR errors that subsequently can inform improvements in EHR design and usability.

Does inadequate EHR design lead to more errors?

While the use of electronic EHRs has improved the coordination and quality of patient care, poor EHR design, usability, and implementation has also led to unintended consequences. These flaws may result in adverse events or other medical errors that can harm patients. While the vast majority of potential errors are either caught by safety checks built into EHRs or identified by staff, there is no system in place to retrospectively identify and review those potential errors, which could inform improvements in EHR design and usability

“As a practicing clinician, you see themes of errors that happen when using EHRs. They happen over and over again and more than 99 percent of the time nothing bad happens. They don’t actually reach the patient, but they keep on happening. And if they are not getting studied and understood, then we are never going to be able to design them out of the system.”
- Dr. Hettinger

Applying the flight black box concept to healthcare

Dr. Aaron (Zach) Hettinger and a research team at the MedStar National Center for Human Factors in Healthcare wanted to find a way to identify near misses in healthcare. They looked to another high-risk industry—aviation—that has examined why and what types of errors occur, and then put in systems to capture the context around errors. Could the concept of a “black box,” a flight data recorder that has led better system design and preventing predictable errors in aviation, be applied to healthcare to improve EHR systems and patient safety?

By applying an innovative health IT black box methodology, a video screen that captures clinical interaction in the EHR, including mouse movement and keystrokes, errors were retrospectively analyzed to understand factors that led to those errors. The approach successfully identified and allowed review of multiple EHR-based errors, informing recommendations for EHR design guidance.

Fighting EHR “bloat”

For example, one analysis done was around acetaminophen (Tylenol) errors involving the wrong route of administration, for instance giving the medication rectally when it was meant to be given by mouth. These ‘wrong route’ errors can be catastrophic in some circumstances and are a focus of patient safety efforts. The researchers found that, in this case, providers had up to 80 different versions of acetaminophen orders to choose from in the EHR that had been added over time by various implementation teams. This had the unintended consequence of showing duplicate and unnecessary orders to the end user, who consistently chose order sentences with a different route of administration from the one they intended. This “EHR bloat”—the addition of new features and minor adjustments to the EHR over time—impacts the usability of the system and may lead to errors. While these additions may not be noticed by informatics teams, end users are susceptible to the errors they cause and may not appreciate the role of the interface in causing the errors.

How small inconsistencies can lead to increased errors

A second example found from the black box analysis, was errors related to the intake form for children versus for adults. On one form, the height was on the left and the weight was on the right, while on the other form, the placement of height and weight was switched. The inconsistency of where this information is entered increased the likelihood that a clinician who sees both pediatric and adult patients, and therefore uses both forms, would enter height and weight incorrectly. “They're doing that 20, 30, 40 times a day, and now all of a sudden they're in what we call automaticity mode, they're going to be more likely to make that error, not because they're bad workers, or they don't care. It's the way that the brain works.” Once identified, Dr. Hettinger and his team made the forms consistent, making an immediate impact on quality and safety.

A valuable tool in increasing patient safety

Dr. Hettinger and his team showed the feasibility and value of creating a methodology and process for a health IT black box to inform EHR design and usability. Seeing the EHR through the eyes of the clinician, at the exact moment that an error occurs, is a powerful tool for change. The value of this tool is amplified when the same error is demonstrated across institutions, providers, and settings that use a common EHR. EHRs have frequently been identified as sources of error and frustration. Yet by studying errors across systems, the researchers are confident implementing active surveillance, like the health IT black box, will result in a collaborative approach to designing systems that facilitate easier and safer care for patients.

This project does not have any related survey.
This project does not have any related story.
This project does not have any related emerging lesson.
Similar to the airline industry’s use of a “black box” that captures actions leading up to a near miss or error, the health IT black box captures mouse movements and keystrokes made by users of EHRs. This allows for a robust analysis of the root cause of EHR errors that subsequently can inform improvements in EHR design and usability.

Project Details - Ended

Summary:

While the use of electronic health records (EHRs) has improved the coordination and quality of patient care, poor EHR design, usability, and implementation has also led to unintended consequences. These flaws may result in adverse events or other medical errors that can harm patients. While the vast majority of potential errors are either caught by safety checks built into EHRs or identified by staff, there is no system in place to retrospectively identify and review those potential errors. A system for reviewing errors could inform improvements in EHR design and usability.

To address this, a research team at the MedStar National Center for Human Factors in Healthcare wanted to find a way to identify near misses. They looked to another high-risk industry-–aviation-–that has examined why and what types of errors occur, and then put in systems to capture the context around errors. The MedStar team hypothesized that a concept of a “black box,” a flight data recorder that has led better system design and preventing predictable errors in aviation, could be applied to healthcare to improve EHR systems and patient safety.

The specific aims of this research were as follows:

  • Identify and characterize potential EHR-facilitated error patterns from clinical EHR data, including wrong-patient, wrong-medication, and wrong-route errors. 
  • Analyze the context of the error through video review and systematically categorize the observed errors to understand what factors lead to EHR-facilitated errors. 
  • Identify the specific EHR design elements facilitating the errors, create data-driven design guidelines based on the most common error-causing elements, and disseminate findings to diverse stakeholders, including providers and EHR developers. 

A retrospective methodology was used to identify EHR errors with the goal of clinical systems optimization. By applying the innovative health IT black box methodology, a video screen that captures clinical interaction in the EHR, including mouse movement and keystrokes, errors were retrospectively analyzed to understand factors that led to those errors. Five error types were explored: wrong-route errors, wrong-weight errors, wrong-sided errors, free-text order errors, and wrong-route-order errors.

The research team found that the approach successfully identified and allowed review of multiple EHR-based errors, informing recommendations for EHR design guidance.

This study showed the feasibility and value of creating a methodology and process for a health IT black box to inform EHR design and usability. Seeing the EHR through the eyes of the clinician, at the exact moment that an error occurs, is a powerful tool for change. The value of this tool is amplified when the same error is demonstrated across institutions, providers, and settings that use a common EHR. EHRs have frequently been identified as sources of error and frustration. Yet by studying errors across systems, the researchers are confident implementing active surveillance, like the health IT black box, will result in a collaborative approach to designing systems that facilitate easier and safer care for patients.