Improving Quality in Cancer Screening: The Excellence Report for Colonoscopy (Oregon)

Project Final Report (PDF, 119.11 KB) Disclaimer

Project Details - Ended

Project Categories

Summary:

This project was designed to evaluate and improve the quality of screening and diagnostic colonoscopies in ambulatory care settings. It was implemented within the Clinical Outcomes Research Initiative (CORI) national consortium. Defined quality measures for colonoscopy were used in individual quality report cards with 15 measures created and made available to participating endoscopists. Quantitative and qualitative studies were performed on the effect of these reports.

The objectives of the project were as follows:

  1. To create a quality report card, the "Excellence Report", from this clinical data demonstrating individual performance on evidence-based quality measures for colonoscopy, including comparisons with other CORI practitioners and nationally published benchmarks.
  2. To determine if providing this quality report card as monthly feedback to endoscopists in ambulatory settings would result in quality improvement as evidenced by improved documentation of and adherence to the quality measures.
  3. To evaluate, using qualitative research techniques, the perceptions by participating endoscopists of this feedback and to search for both unintended consequences and alterations in workflow resulting from the intervention.
  4. To explore and contribute to emerging interoperability standards for quality measurement and to help bring these standards to the Gastrointestinal community.

Data analysis is still pending on the quantitative study, but it is the expectation that no change in compliance with the quality measures will be found. The qualitative study revealed that endoscopists are not only aware of quality measurement recommendations but have changed their practice to meet those recommendations. Most endoscopists felt that they were performing within guidelines and would be willing to share their quality reports, although their distrust of payors was clear. Implementation of the chosen quality measures proved difficult for several reasons, including the difficulty of finding and validating correctness of data from complex clinical systems and the complexity of quality criteria, especially for determining exclusions. Implementation was also limited by the difficulty of retrieving pathology results, whether in narrative or discrete data form, and even in obtaining access to images of pathology result reports.

The project team concluded that clinicians will not need encouragement to participate in quality measurement programs, which are seen as inevitable and even desirable, but that programs will need to emphasize quality measures that are relevant to clinical care and be certain about the accuracy of the information presented.

Improving Quality In Cancer Screening: The Excellence Report For Colonoscopy - 2010

Summary Highlights

  • Principal Investigator: 
  • Funding Mechanism: 
    RFA: HS07-002: Ambulatory and Safety Quality Program: Enabling Quality Measurement through Health Information Technology (EQM)
  • Grant Number: 
    R18 HS 017017
  • Project Period: 
    August 2007 – August 2010,Including No-Cost Extension
  • AHRQ Funding Amount: 
    $616,207
  • PDF Version: 
    (PDF, 370.53 KB)


Target Population: Adults, Cancer

Summary: Colorectal cancer is the second leading cause of cancer-related deaths in the United States. Colonoscopy is used increasingly for primary screening and to evaluate positive screening tests, but the effectiveness of colonoscopy depends on providing high-quality examinations with few complications. This initiative is designed to evaluate and improve the quality of screening and diagnostic colonoscopies in ambulatory care settings by presenting the Excellence Report, a quality report card to clinicians.

Using the Clinical Outcomes Research Initiative (CORI) software application and the National Endoscopic Database, the project developed and tested the Excellence Report. The Excellence Report is a quality report card for gastrointestinal (GI) endoscopy that focuses on nationally recognized quality process measures for colonoscopy. Using a clustered-randomized trial design, the project staff sent CORI-affiliated clinicians working in ambulatory care centers or offices monthly reports of their quality measures through a secure Web site along with comparisons to other CORI clinicians and national benchmarks. The effects of reporting quality measures data, adherence to quality recommendations, and durability of changes upon discontinuing the reports, were measured. Concurrently, field observations and interviews were performed with a representative sample of clinicians receiving the Excellence Report. The objectives of these interviews were: to understand clinician perceptions of reliability and validity of the data presented; to understand clinician acceptance of the quality initiative; and to look for effects on workflow and any unintended consequences of the Excellence Report. In addition, this project, along with GI specialty societies, coordinated a series of four Webinars with representatives from GI reporting software vendors, imaging system vendors, and GI pathology laboratories. The Webinars provided information on the development of a sustainable and standards-based architecture that will allow the Excellence Report to expand beyond CORI.

Specific Aims:

  • Create the Excellence Report, a quality report card of individual performance on quality measures, and present this as monthly feedback to the ambulatory care providers of CORI. (Achieved)
  • Measure the effect of the Excellence Report on individual performance in adherence to the recommended quality measures for colonoscopy. (Achieved)
  • Perform a qualitative assessment of the effect of the Excellence Report on providers, study the acceptance of the individual Excellence Report and effects on workflow, and identify any unintended consequences. (Achieved)
  • Lead an industry consortium comprised of endoscopy reporting software vendors, pathology laboratories, and endoscopy imaging vendors to develop industry-wide standards for the exchange of data on colonoscopy quality measures. (Achieved)

2010 Activities: The primary focus for study staff during this period was the completion of data analysis and the development of manuscripts for submission to peer review journals.

Grantee’s Most Recent Self-Reported Quarterly Status (as of August 2010): The project is complete with all major aims achieved and project spending on target.

Impact and Findings: Overall changes in compliance with all quality measures were small, and the direction of change for each quality measure was not always towards improvement. All measures depend upon adequate documentation and there is a wide variation between endoscopists in their rate of documentation. American Society of Anesthesiologist classification, for example, was documented from 0 to 100 percent of the time in both intervention and control groups. If endoscopists who documented 0 percent of the time are excluded, however, a value is documented an average of 93 percent of the time. It is possible that those clinicians who did not document in CORI are collecting this information in other documentation, such as in a separate anesthesia record. This same variability in completeness of documentation was seen with all measures.

Analysis of significance of changes was performed in two measures, chosen because of their significance in quality patient care. For these two measures a general linear mixed model (GLMM) was used to test for differences in the quality measures pre- and post- intervention between control and intervention groups. The data consisted of summary data (proportion of exams that met the quality measure) on a per endoscopist level. The GLMM method allowed researchers to investigate a nested structure – communities within each treatment arm and then providers within communities (and two time points for providers). This allowed control for potential correlation between observations from the same community and additional correlation between observations from the same provider. As part of the analysis, the project team also tested the difference between pre- and post-intervention depending on treatment arm. For both models, community and the interaction between pre- and post-intervention arms (examining if pre- vs. post-intervention differences depended on treatment arm) were not found to be significant and did not remain in the final models.

The qualitative study revealed six themes: workflow, organizational structure and accreditation, temporal issues, integration issues, value of measures and reports, and the need for data of high quality. Each theme is discussed below:

Workflow. Staff members at each of the five sites designed clinic workflow for colonoscopy procedures somewhat differently, but there were many similarities in how and when clinicians used CORI. The second aspect to workflow was accessing the Excellence Reports. The clinicians who had looked at the reports felt that they might access the reports if they were either a novice or if they were trying to improve their ratings on a particular measure. Most physicians felt that monthly access is too frequent and quarterly might be more useful. They felt that sending the reports to clinic managers might be more useful than requiring access by physicians.

Organizational Structure and Accreditation. There was great variation between a small clinic without Ambulatory Surgery Center (ASC) accreditation and the larger clinics that were accredited. The sites with ASC accreditation tended to be more motivated to measure quality because of certification requirements. In addition, some specialty board recertification for the GI physicians required quality measurement which may have influenced physician willingness to measure quality.

Temporal Issues. Clinicians are concerned about administrative burden to such an extent that, when asked what they thought about pay for performance incentives, they always expressed doubt that the incentive offered would be worth the additional manpower needed for reporting. Temporal issues also arose when physicians discussed measures they would most like. For instance, gastroenterologists would like to see a measurement around adenoma counts which require additional data entry post-procedure once the final pathology report is received. Only one site was entering such after-the-procedure information.

Integration Issues. More complete quality measurement could be accomplished if CORI, the different electronic medical records, and the imaging systems could be interoperable. Two sites double enter information into both the electronic medical records and CORI. Most use separate imaging systems, though one can import the images into CORI. At one site, the GI physicians have an ambulatory clinic using CORI and also practice at a hospital next door that uses CORI; however, the data in the two CORI systems are not integrated.

Value of Measures and Reports. Staff members at the sites that have designated quality assurance physicians and Accreditation Association for Ambulatory Health Care accreditation seem highly motivated and even excited about the Excellence Reports and measurement in general. Systems need to be designed so that the data entry burden is not large. Many noted that insurance companies now want to see outcomes data. In addition, several interviewees reported that patients are starting to ask how the clinicians rate on some measures.

The Need for Data of High Quality. The measures that come out of the system and that can be reported on are only as good as the data that enter the system.

In addition, differing viewpoints were detected which are described below:

Specialists and Other Providers: At nearly every site, the GI physicians’ view of quality measures is very different from that of general surgeons or family practitioners. The GI physicians are proud to be knowledgeable about the latest evidence and on the forefront of putting this evidence into practice. One point of contention with those who are not specialists is followup protocols. The specialists are confident that quality reports will be in their favor.

Specialists and Hospitals: Another differing view is that of the hospital. There seems to be some competition between hospitals and ambulatory clinics and, as researchers were told, hospitals in some States have lobbied to limit the number of clinics that can be licensed to perform these procedures.

Strategic Goal: Develop and disseminate health IT evidence and evidence-based tools to improve health care decisionmaking through the use of integrated data and knowledge management.

Business Goal: Synthesis and Dissemination

Improving Quality in Cancer Screening: The Excellence Report for Colonoscopy - Final Report

Citation:
Logan JR. Improving Quality in Cancer Screening: The Excellence Report for Colonoscopy - Final Report. (Prepared by the Oregon Health & Science University under Grant No. R18 HS017017). Rockville, MD: Agency for Healthcare Research and Quality, 2009. (PDF, 119.11 KB)

The findings and conclusions in this document are those of the author(s), who are responsible for its content, and do not necessarily represent the views of AHRQ. No statement in this report should be construed as an official position of AHRQ or of the U.S. Department of Health and Human Services.
Principal Investigator: 
Document Type: 
Technology: 
Population: 
Medical Condition: 

The Excellence Report: Quality Measures for Colonoscopy

PDF: The Excellence Report: Quality Measures for Colonoscopy (PDF, 192.03 KB)

FAQs: The Excellence Report: Colonoscopy Quality Measures in Ambulatory Care

PDF: FAQs: The Excellence Report: Colonoscopy Quality Measures in Ambulatory Care (PDF, 80.17 KB)

Quality Measures for Colonoscopy, CORI v4

PDF: Quality Measures for Colonoscopy, CORI v4 (PDF, 72.1 KB)

Quality Measures for Colonoscopy, CORI v4 (2)

PDF: Quality Measures for Colonoscopy, CORI v4 (2) (PDF, 41.63 KB)
This project does not have any related survey.
This project does not have any related project spotlight.
This project does not have any related survey.
This project does not have any related story.
This project does not have any related emerging lesson.