Improving Quality In Cancer Screening: The Excellence Report For Colonoscopy - 2010

Target Population: Adults, Cancer

Summary: Colorectal cancer is the second leading cause of cancer-related deaths in the United States. Colonoscopy is used increasingly for primary screening and to evaluate positive screening tests, but the effectiveness of colonoscopy depends on providing high-quality examinations with few complications. This initiative is designed to evaluate and improve the quality of screening and diagnostic colonoscopies in ambulatory care settings by presenting the Excellence Report, a quality report card to clinicians.

Using the Clinical Outcomes Research Initiative (CORI) software application and the National Endoscopic Database, the project developed and tested the Excellence Report. The Excellence Report is a quality report card for gastrointestinal (GI) endoscopy that focuses on nationally recognized quality process measures for colonoscopy. Using a clustered-randomized trial design, the project staff sent CORI-affiliated clinicians working in ambulatory care centers or offices monthly reports of their quality measures through a secure Web site along with comparisons to other CORI clinicians and national benchmarks. The effects of reporting quality measures data, adherence to quality recommendations, and durability of changes upon discontinuing the reports, were measured. Concurrently, field observations and interviews were performed with a representative sample of clinicians receiving the Excellence Report. The objectives of these interviews were: to understand clinician perceptions of reliability and validity of the data presented; to understand clinician acceptance of the quality initiative; and to look for effects on workflow and any unintended consequences of the Excellence Report. In addition, this project, along with GI specialty societies, coordinated a series of four Webinars with representatives from GI reporting software vendors, imaging system vendors, and GI pathology laboratories. The Webinars provided information on the development of a sustainable and standards-based architecture that will allow the Excellence Report to expand beyond CORI.

Specific Aims:

• Create the Excellence Report, a quality report card of individual performance on quality measures, and present this as monthly feedback to the ambulatory care providers of CORI. (Achieved)
• Measure the effect of the Excellence Report on individual performance in adherence to the recommended quality measures for colonoscopy. (Achieved)
• Perform a qualitative assessment of the effect of the Excellence Report on providers, study the acceptance of the individual Excellence Report and effects on workflow, and identify any unintended consequences. (Achieved)
• Lead an industry consortium comprised of endoscopy reporting software vendors, pathology laboratories, and endoscopy imaging vendors to develop industry-wide standards for the exchange of data on colonoscopy quality measures. (Achieved)

2010 Activities: The primary focus for study staff during this period was the completion of data analysis and the development of manuscripts for submission to peer review journals.

Grantee’s Most Recent Self-Reported Quarterly Status (as of August 2010): The project is complete with all major aims achieved and project spending on target.

Impact and Findings: Overall changes in compliance with all quality measures were small, and the direction of change for each quality measure was not always towards improvement. All measures depend upon adequate documentation and there is a wide variation between endoscopists in their rate of documentation. American Society of Anesthesiologist classification, for example, was documented from 0 to 100 percent of the time in both intervention and control groups. If endoscopists who documented 0 percent of the time are excluded, however, a value is documented an average of 93 percent of the time. It is possible that those clinicians who did not document in CORI are collecting this information in other documentation, such as in a separate anesthesia record. This same variability in completeness of documentation was seen with all measures.

Analysis of significance of changes was performed in two measures, chosen because of their significance in quality patient care. For these two measures a general linear mixed model (GLMM) was used to test for differences in the quality measures pre- and post- intervention between control and intervention groups. The data consisted of summary data (proportion of exams that met the quality measure) on a per endoscopist level. The GLMM method allowed researchers to investigate a nested structure – communities within each treatment arm and then providers within communities (and two time points for providers). This allowed control for potential correlation between observations from the same community and additional correlation between observations from the same provider. As part of the analysis, the project team also tested the difference between pre- and post-intervention depending on treatment arm. For both models, community and the interaction between pre- and post-intervention arms (examining if pre- vs. post-intervention differences depended on treatment arm) were not found to be significant and did not remain in the final models.

The qualitative study revealed six themes: workflow, organizational structure and accreditation, temporal issues, integration issues, value of measures and reports, and the need for data of high quality. Each theme is discussed below:

Workflow. Staff members at each of the five sites designed clinic workflow for colonoscopy procedures somewhat differently, but there were many similarities in how and when clinicians used CORI. The second aspect to workflow was accessing the Excellence Reports. The clinicians who had looked at the reports felt that they might access the reports if they were either a novice or if they were trying to improve their ratings on a particular measure. Most physicians felt that monthly access is too frequent and quarterly might be more useful. They felt that sending the reports to clinic managers might be more useful than requiring access by physicians.

Organizational Structure and Accreditation. There was great variation between a small clinic without Ambulatory Surgery Center (ASC) accreditation and the larger clinics that were accredited. The sites with ASC accreditation tended to be more motivated to measure quality because of certification requirements. In addition, some specialty board recertification for the GI physicians required quality measurement which may have influenced physician willingness to measure quality.

Temporal Issues. Clinicians are concerned about administrative burden to such an extent that, when asked what they thought about pay for performance incentives, they always expressed doubt that the incentive offered would be worth the additional manpower needed for reporting. Temporal issues also arose when physicians discussed measures they would most like. For instance, gastroenterologists would like to see a measurement around adenoma counts which require additional data entry post-procedure once the final pathology report is received. Only one site was entering such after-the-procedure information.

Integration Issues. More complete quality measurement could be accomplished if CORI, the different electronic medical records, and the imaging systems could be interoperable. Two sites double enter information into both the electronic medical records and CORI. Most use separate imaging systems, though one can import the images into CORI. At one site, the GI physicians have an ambulatory clinic using CORI and also practice at a hospital next door that uses CORI; however, the data in the two CORI systems are not integrated.

Value of Measures and Reports. Staff members at the sites that have designated quality assurance physicians and Accreditation Association for Ambulatory Health Care accreditation seem highly motivated and even excited about the Excellence Reports and measurement in general. Systems need to be designed so that the data entry burden is not large. Many noted that insurance companies now want to see outcomes data. In addition, several interviewees reported that patients are starting to ask how the clinicians rate on some measures.

The Need for Data of High Quality. The measures that come out of the system and that can be reported on are only as good as the data that enter the system.

In addition, differing viewpoints were detected which are described below:

Specialists and Other Providers: At nearly every site, the GI physicians’ view of quality measures is very different from that of general surgeons or family practitioners. The GI physicians are proud to be knowledgeable about the latest evidence and on the forefront of putting this evidence into practice. One point of contention with those who are not specialists is followup protocols. The specialists are confident that quality reports will be in their favor.

Specialists and Hospitals: Another differing view is that of the hospital. There seems to be some competition between hospitals and ambulatory clinics and, as researchers were told, hospitals in some States have lobbied to limit the number of clinics that can be licensed to perform these procedures.

Strategic Goal: Develop and disseminate health IT evidence and evidence-based tools to improve health care decisionmaking through the use of integrated data and knowledge management.

Business Goal: Synthesis and Dissemination