Using Precision Performance Measurement to Conduct Focused Quality Improvement - 2010

Principal Investigator
Funding Mechanism
RFA: HS07-006: Ambulatory Safety and Quality Program: Improving Quality Through Clinician Use of Health Information Technology (IQHIT)
Grant Number
R18 HS 017163
Project Period
September 2007 – August 2011, Including No-Cost Extension
AHRQ Funding Amount
$1,199,415
Summary Status as of
December 2010


Target Population: Adults, Chronic Care*, Diabetes, Heart Disease, Hypertension

Summary: Measures that utilize data collected for administrative use, such as billing data, inevitably have inaccuracies at the individual patient level. A quality measure may be recorded as not having been met because a patient was incorrectly considered to be eligible or refused the intervention, or because the appropriate data were not captured. As a result of these limitations, clinicians may in truth reach their quality benchmark targets, but automatic reporting of measures pulled from administrative data fail to accurately reflect this. Imprecise measurement methods can never be the foundation for a health care system that delivers near 100 percent high-quality care for chronic disease care and prevention. Quality measurement needs to be embedded within electronic health record (EHR) systems and become dynamic, accurate, and detailed to support the highest level of care possible for all patients.

This project creates systems that allow clinicians to capture reasons for not providing care as part of point-of-care clinical decision support reminder systems, improve data quality, and seamlessly link data to practice-level quality improvement programs and point-of-care interventions. The project uses previously developed quality measurement programs that examine EHR data to measure quality of care for coronary artery disease, heart failure, diabetes, hypertension, and preventive services. This study began at a large academic internal medicine practice and is now being implemented in four community practices that use the same Certification Commission for Health Information Technology-certified EHR, produced by Epic.

Exception codes are being introduced into the EHR for 18 national quality measures. Data are extracted from the EHR every month to assess changes in the primary outcome: the proportion of eligible patients who do not satisfy a measure and do not have any exclusion criteria documented. The statistical significance of changes will be assessed with time series analysis. In addition, physicians will be repeatedly surveyed on their attitudes toward the interventions described in the aims listed below. Outcomes of the quality improvement activities will be monitored along with the costs of the intervention. This study will produce computerized tools and educational materials that can be provided to more than 1,000 sites that use the Epic EHR ambulatory product.

Specific Aims:
  • Integrate simple, standard ways for clinicians to document patient reasons or medical reasons for why quality measures are not met and assess the use of these exception codes, the impact of exception reporting on measured levels of quality, and the impact of using these codes on physician satisfaction and self-reported efficiency. (Ongoing)
  • Use the exception codes (patient reasons and medical reasons) that clinicians enter to target three forms of quality improvement, including: 1) peer review of all medical reasons for not adhering to guidelines followed by academic detailing if a clinician enters an unjustified reason for not following guidelines; 2) counseling for patients whose physician enters an exclusion code stating that the patient cannot afford a needed medication to determine ways of overcoming barriers; and 3) educational outreach to all patients who refuse recommended interventions, including mailing of plain-language health education materials or DVDs. (Ongoing)
  • Provide clinicians with highly accurate information on patients’ quality deficits immediately prior to their visit as part of routine workflow, and assess whether this intervention increases provision of recommended therapies and tests and documentation of exclusion codes. (Ongoing)

2010 Activities: Implementation at the Northshore site significantly progressed. The site was able to implement the clinical decision support and reminder tools in the EHR for select conditions, and generated individual physician and group-level quality reports that included the data on entered exceptions. Data has been generated for the time series studies to analyze changes in quality of care over the course of the study as measured by increases in patients receiving the service, documentation of exceptions, or a combination of both. Further preparation of the data for the time series studies continued beyond the end of the year. Initial reviews of the validity of the entered medical exceptions have been completed which have shown a high level of validity for entered exceptions, only slightly below that seen at the Northwestern site.

Analyses of the effects of the pre-encounter quality deficit reminder system were completed. Although quality of care continued to improve during this second year of the intervention, the pre-encounter notification system did not appear to engage those physicians who were not frequently using the electronic clinical decision support tools. However, the physician survey suggested that doctors liked it, so it has been continued. The study team began to write up these results.

Grantee’s Most Recent Self-Reported Quarterly Status (as of December 2010): The project’s progress is reported as completely on track and is meeting 100 percent of its milestones on time. Project spending is roughly on target.

Preliminary Impact and Findings: For the first aim, the primary outcome of ten measures significantly improved more rapidly the year after implementation than during the prior year. For four other measures, quality improved, but the rate of improvement did not differ significantly from the year prior to the intervention. One measure improved at a significantly slower rate, and the performance of mammography declined due to new barriers to access at the study site. Improvements resulted from increases in patients receiving the service, documentation of exceptions, or a combination of both. By the end of the first year, for five drug prescribing measures, over half of physicians achieved 100 percent performance.

For the second aim, 6.5 percent of the quality reviews identified an issue requiring feedback from an investigator to a clinician, who then entered a medical exception. In patient outreach, the majority of patients did not want to talk about their refusal. Of all patients, 13.5 percent eventually completed a test or took a medication they originally declined.

Strategic Goal: Develop and disseminate health IT evidence and evidence-based tools to improve health care decisionmaking through the use of integrated data and knowledge management.

Business Goal: Implementation and Use

*AHRQ Priority Population.