Evaluating Smart Forms and Quality Dashboards in an Electronic Health Record (EHR) (Massachusetts)

Project Final Report (PDF, 86.44 KB) Disclaimer

Evaluating Smart Forms and Quality Dashboards in an EHR - 2009

Summary Highlights

  • Principal Investigator: 
  • Funding Mechanism: 
    RFA: HS04-012: Demonstrating the Value of Health Information Technology (THQIT)
  • Grant Number: 
    R01 HS 015169
  • Project Period: 
    September 2004 – September 2009, Including No-Cost Extension
  • AHRQ Funding Amount: 
  • PDF Version: 
    (PDF, 442.25 KB)

Target Population: Acute Respiratory Infections, Adults, Chronic Care*, Diabetes, Heart Disease

Summary: This project evaluated the potential for improving the management of patients with acute and chronic medical conditions through the creation of clinical decision support (CDS) tools integrated with clinical documentation workflow, and the provision of physician performance feedback on guideline compliance and quality benchmarks in an electronic health record (EHR). The project sought to increase patient safety and quality and address the combined needs of clinical workflow support and decision support with innovative EHR technology.

The project examined two EHR-based interventions to determine if their use in chronic and acute conditions significantly improves patient clinical outcomes. The first intervention focused on the impact of a forms-based, clinical documentation approach to decision support, called Smart Forms (SFs). SFs incorporate on-screen patient chart reviews, effective coded data capture, note generation, and actionable CDS at the point-of-care to help physicians manage chronic and acute disease, including coronary artery disease (CAD), diabetes mellitus (DM), and acute respiratory infection (ARI). Patient education materials on self-management skills can also be generated from the same screen. The second EHR-based intervention, Quality Dashboards (QDs), provide clinician-specific performance reports on guideline compliance and quality benchmark achievement. QDs help physicians, medical directors, and practice leaders by reporting adherence to recommended clinical guidelines. The QD graphical displays allow for same-practice physician comparison and local and national benchmark comparison. The SF and QD are both integrated with the Longitudinal Medical Record (LMR), the EHR system that was developed by Partners HealthCare System (PHS).

The two interventions were implemented in more than 20 primary care practices in the PHS and more than 400 clinician study participants. To evaluate the SF and QD technology the following randomized, controlled trials (RCTs) were conducted:

  • Acute Respiratory Infection Smart Form Randomized Controlled Trial (ARI SF RCT)
  • Coronary Artery Disease/Diabetes Mellitus Smart Form Randomized Controlled Trial (CAD/DM SF RCT)
  • Acute Respiratory Infection Quality Dashboard Randomized Controlled Trial (ARI QD RCT)
  • Coronary Artery Disease Quality Dashboard Randomized Controlled Trial (CAD QD RCT)

In each RCT, PHS-affiliated primary care practices were randomized to follow established protocol, the SF-enhanced protocol, or the SF plus QD enhanced protocol. Data collected from the RCTs and physician surveys were used to answer the following questions: 1) Can a usable EHR-based intervention that provides CDS in the context of clinical documentation workflow and that integrates population-based performance feedback to the physician be developed?; and 2) Do these interventions improve the quality of clinical documentation, capture of key clinical data for CDS and quality assessment, and compliance with best practice guidelines?

Specific Aims:

  • Design and implement an integrated documentation-based CDS and physician feedback system provided in an EHR to improve the management of patients with acute and chronic medical conditions. (Achieved)
  • Determine the effectiveness of a documentation-based CDS tool and physician feedback with respect to documentation and the clinical management of patients with CAD and ARI. (Achieved)
  • Assess the perceived value of QDs by clinicians and if they impact SFs’ compliance with best practices in ARI and CAD. (Achieved)

2009 Activities: Analysis of surveys related to the CAD/DM SF RCT was completed in 2009, while data analysis of the ARI QD RCT was ongoing in 2010. The CAD QD RCT was completed in March 2009 and encompassed 15 primary care practices involving approximately 350 clinicians.

Grantee’s Most Recent Self-Reported Quarterly Status (as of September 2009): The ARI and CAD QD RCTs were completed in 2009. Data analysis was ongoing in 2010.

Impact and Findings: A documentation-based CDS tool and a physician feedback system can be designed and implemented in an outpatient clinical setting. Reported use and evaluation of usability suggest that including such a CDS tool into clinical practice facilitates common tasks such as documentation and order-entry.

The results of the ARI study demonstrated that use of SFs decrease instances of antibiotic prescribing for non-antibiotic-appropriate diagnoses, suggesting better adherence to clinical guidelines on the part of the clinician. The results of the CAD/DM study suggest that using SFs as a CDS tool provide for more up-to-date documentation of clinical data, particularly of patients with chronic conditions. Although the QD studies are pending further analysis, the preliminary results indicate the potential for an EHR-based QD system to provide additional benefit beyond a CDS tool like SF in the management of patients with CAD/DM and ARI.

Although the data analysis is ongoing, a trend has emerged from the study results up to this point. Overall, use of SFs and QDs as a part of clinical decision support correlates with better adherence to the clinical guidelines within the areas described. Also, most users found these tools intuitive to use, easy to integrate into clinicians’ workflow, and beneficial in terms of quality of patient care.

Strategic Goal: Develop and disseminate health IT evidence and evidence-based tools to support patient-centered care, the coordination of care across transitions in care settings, and the use of electronic exchange of health information to improve quality of care.

Business Goal: Knowledge Creation

*AHRQ Priority Population.

Evaluating Smart Forms and Quality Dashboards in an Electronic Health Record (EHR) - Final Report

Middleton B. Evaluating Smart Forms and Quality Dashboards in an Electronic Health Record (EHR) - Final Report. (Prepared by Brigham and Women's Hospital under Grant No. R01 HS015169). Rockville, MD: Agency for Healthcare Research and Quality, 2009. (PDF, 86.44 KB)

The findings and conclusions in this document are those of the author(s), who are responsible for its content, and do not necessarily represent the views of AHRQ. No statement in this report should be construed as an official position of AHRQ or of the U.S. Department of Health and Human Services.
Principal Investigator: 
Document Type: 
This project does not have any related event.
This project does not have any related resource.
This project does not have any related survey.
This project does not have any related project spotlight.
This project does not have any related survey.

AHRQ Project Seeks To Cure What Ails Electronic Health Records

Blackford Middleton, M.D., M.P.H., M.Sc.As anyone who has ever tried to implement an electronic health record (EHR) in clinical practice will tell you, if physicians and others on the frontline of care don't see the value, you won't get very far.

The demands of modern clinical practice require doctors to keep up on the latest in best practices. At the same time, doctors are under increasing pressure to meet certain care measurements for patients under pay-for-performance programs. Still, studies have found that patients only get the recommend care they need 50 percent of the time. And many EHRs are limited in their ability to provide physician performance feedback on quality.

But a new projected funded by the Agency for Healthcare Research and Quality (AHRQ) aims to change that. The project, led by Blackford Middleton, M.D., director of clinical informatics research & development for the Partners Healthcare System, and researchers at Brigham and Women's Hospital and Harvard Medical School, is examining whether information technology tools that provide both clinical-decision support and population-based performance feedback will increase the value of EHRs to clinicians while improving patient safety and quality for patients with acute respiratory illness or coronary artery disease.

In addition, researchers want to see whether these tools can have an impact on health care costs by eliminating duplicate tests and procedures. To make their EHRs more useful, researchers began implementing "smart forms" and "quality dashboards" at clinical practices in the Boston area affiliated with Massachusetts General Hospital and Brigham and Women's Hospital.

Smart forms help physicians to document relevant clinical information and integrate best practice information and ordering capabilities into care. For example, while documenting a patient encounter, the system reminds a doctor that the patient should be taking a particular medication. If appropriate, the doctor can instantly order the medication for the patient, update the EHR, and document the order in the visit note in one step.

"This is a new kind of decision support," says Jeffrey Linder, M.D., a principal investigator for the project. "The whole idea is that ithelps doctors do the right thing easily."

The quality dashboard, which works hand-in-hand with the smart forms, is a system that provides feedback to physicians on their patient population and gives them the ability to see how they are performing on certain measures of quality. The clinician can even compare how they are performing on these measures in relation to other doctors in the clinic or against national benchmarks for quality. The physician can then drill down from the population view and see individual information on patients through the smart forms.

Both Linder and Jeffrey Schnipper, M.D., another principal investigator on the project, say these tools help integrate performance measures into EHRs, making them more valuable. They also provide more accurate quality data. Most information on clinical quality is currently based on claims data, which are retrospective and don't always provide the full picture, Linder and Schnipper say.

"We'd like to push the envelope on how quality reporting is done," the researchers say.

Currently, the dashboard and smart form for acute respiratory patients has been introduced to about 1,000 doctors in 10 practices. The coronary artery disease/diabetes smart form is currently being rolled out to a similar number of practices, while the dashboard for coronary artery disease is being pilot tested in two practices to 20 physicians.

Thus far, researchers say that for technology to be accepted, it has to fit clinician's existing workflow. Introducing new technology will mean a temporary reduction in efficiency. The key is to make change as painless as possible while clearly explaining the benefits of the technology, Schnipper says.

With that in mind, Linder and Schnipper plan to integrate a new "population management" tool into the dashboard system that will allow clinicians to send batch letters to patients, reminding them that they are due for a scheduled test, or sending themuseful information about a medication they are taking. Ordering and documenting the test would also be done instantly.

"Physicians are so crunched for time. We hope that decision support is medicine that goes down well," Linder says. 

This project does not have any related emerging lesson.


Computer-based clinical decision support (CDS) systems are health information tools that combine education, physician participation, and feedback via reminders. These technologies have the potential to change physician behavior at the precise time that clinical decisions are being made. However, such systems are still not used broadly, and the full potential of CDS systems and tools remains to be tested.

Issues of usability and integration into the clinicians' workflow are two key barriers to the effectiveness of CDS tools. One potential solution under development at Partners HealthCare is to integrate decision support into clinical documentation templates or Smart Forms (SFs), facilitating clinical decision support, ordering, patient education, and documentation in a single step. These SF tools can also facilitate the acquisition of key quality data that can then be presented in a Quality Dashboard (QD), a tool that tracks statistical data about patient care and presents a comparative analysis across various groups to the user. If linked to the electronic health record (EHR), QDs can enhance feedback by providing actionable, population-based information on quality of care, facilitate adherence to guidelines, and identify patients most in need of attention. To date, very few EHRs have developed such features and functions.

The purpose of this project was to design and implement SF and QD tools and to assess their impact on guideline adherence in three clinical areas: acute respiratory tract infections (ARI), coronary artery disease (CAD), and diabetes mellitus (DM). The study encompassed 27 primary care clinics and over 400 clinicians associated with Bringham and Women's and Massachusetts General Hospital, all part of the Partners HealthCare System.

Four randomized, controlled trials were conducted, preceded by smaller scale pilot studies. The main study compared usual care to use of SF alone, and to use of SF plus QD. Outcome measures were compared between the intervention and control practices. The ARI SF study revealed a small but significant difference in antibiotic prescribing rates, with clinicians prescribing antibiotics to 43 percent of ARI patients in control clinics and 39 percent of ARI patients in intervention clinics. CAD/DM SF and CAD/DM QD data are being analyzed. Survey results indicate that exposure to new CDS tools may marginally increase adherence to clinical guidelines. In addition, most users found the tools intuitive to use, easy to integrate into clinicians' workflow, and beneficial in terms of quality of patient care.