Ensuring Safe Performance of Electronic Health Records (Massachusetts)

Project Final Report (PDF, 633.99 KB) Disclaimer

Project Details - Ongoing

Project Categories


Electronic health records (EHRs) are rapidly being deployed across the U.S., but it is uncertain to what extent they will result in desired benefits with respect to safety or quality of patient care. In addition, hospitals have many degrees of freedom in implementation of EHRs, especially with respect to what decision support they implement.

To address these issues, the investigators collaborated to build the Leapfrog Computerized Provider Order Entry (CPOE)/EHR tool, which evaluates the safety performance of EHRs after deployment, with a particular focus on high impact patient safety and medication safety problems. It is essentially a “flight simulator” for EHRs with CPOE, in that hospitals download simulated patients, and then enter simulated predefined groups of orders, and record whether critical decision support appears in response to these scenarios. The simulated orders used in the test include orders that have actually killed patients in the U.S. as well as many that have caused actual harm to patients. After taking the test, hospitals get immediate feedback on their performance in high impact, high prevalence safety areas and – for the subset of potentially fatal orders in the test – hospitals are given the actual list so that they can put key decision support in place.

In 2013, more than 800 hospitals in the U.S. used this tool to self-assess their operational EHR systems for safety performance. Key findings have been that there is great variability in how hospitals score with this tool, which is largely independent of vendor, but that hospitals appear to improve their performance over time after taking the test on repeated occasions, for both potentially fatal and non-fatal orders.

For the current project, investigators will refine and further develop the Leapfrog CPOE/EHR test using the existing Web-based testing approach, including updates to the inpatient version of the test, so that the test is compatible with the latest versions of the leading EHR vendor products and the latest hospital formularies. In addition a new database will be created to host and administer the test. The investigators will then work with four study hospitals using four different leading major vendor applications to refine the test further. In addition, they will track how many hospitals nationally are using the system, how they perform on the test, and also the extent to which they improve with time.

The specific aims of the project are to:

  • Measure the national progress on test performance by hospitals in inpatient EHRs in key domains, both in a cohort of hospitals which has taken the test serially, and then in hospitals overall. 
  • Work to ensure that inpatient EHRs improve safety by updating a widely used existing test. 
  • Improve the test in four test sites to cover additional new high impact safety clinical domains refine them in four health systems, representing four of the leading EHR vendors, and then release them for national use through the Leapfrog Group.
This project does not have any related annual summary.

Ensuring Safe Performance of Electronic Health Records - Final Report

Bates D. Ensuring Safe Performance of Electronic Health Records - Final Report. (Prepared by the Brigham and Women's Hospital under Grant No. R01 HS023696). Rockville, MD: Agency for Healthcare Research and Quality, 2019. (PDF, 633.99 KB)

The findings and conclusions in this document are those of the author(s), who are responsible for its content, and do not necessarily represent the views of AHRQ. No statement in this report should be construed as an official position of AHRQ or of the U.S. Department of Health and Human Services. (Persons using assistive technology may not be able to fully access information in this report. For assistance, please contact Corey Mackison).
Principal Investigator: 
Document Type: 
This project does not have any related resource.
This project does not have any related survey.
This project does not have any related project spotlight.
This project does not have any related survey.
This project does not have any related story.
This project does not have any related emerging lesson.