Closing the Feedback Loop to Improve Diagnostic Quality - 2011

Principal Investigator
Funding Mechanism
RFA: HS07-002: Ambulatory Safety and Quality Program: Enabling Quality Through Health Information Technology (EQM)
Grant Number
R18 HS 017060
Project Period
November 2007 - August 2011
AHRQ Funding Amount
$998,509

Summary: The inpatient setting allows alterative diagnoses to be considered whenever there is a lack of response to therapy or when an adverse event occurs. Determining whether a diagnosis is correct in an outpatient setting may be difficult because patients with inaccurate diagnoses may fail to follow up, get better on their own despite the inaccurate diagnosis, or seek care elsewhere, so that the original provider never learns of the error. Indeed, a "correct" diagnosis may not be discovered until a later date when a biopsy, autopsy, hospital stay, or adverse event occurs and establishes a disparate diagnosis.

This project sought to track outcomes of initial diagnosis and to provide that information to clinicians to give feedback and the opportunity to revise initial diagnoses. The assessment of outcomes was assisted by the involvement of patients. The metric for the quality of the diagnosis was whether the patient's condition resolved in a timely and appropriate manner, or whether - as the result of feedback the initial diagnosis - was modified in a timely manner.

The project developed automated processes for proactive followup and ongoing rapid feedback to physicians in two types of outpatient settings: 1) three ambulatory clinics (the University of Alabama at Birmingham [UAB]-Huntsville Family Practice; United Cerebral Palsy Clinic [UCP]; and the UAB-HIV Clinic); and 2) an emergency department (ED) (Shands-Jacksonville). The ambulatory sites each used different electronic health records (EHRs). UCP used the WorldVistA EHR, UAB used Touchworks EHR, and UAB-HIV used a proprietary EHR. The ED site used the McKesson Horizon Patient Folder and a proprietary ED system that provided a computer-generated paper template that was customized to the patient's chief complaint.

Different interventions were used at each type of site. The clinic site intervention was an interactive voice response (IVR) system that collected followup data for a feedback report to physicians on patient health status and medication adherence. The feedback report used an interface between the EHR and a database that can be integrated with a variety of systems. The ED intervention was an automated followup and feedback report to the ED physicians on the final diagnoses of patients who were admitted to the hospital as compared to their initial ED diagnoses.

Outcome measure included providers' responses to the feedback; satisfaction with the process; its impact on diagnostic and therapeutic quality; response to use of the IVR and ED feedback systems; and use of the feedback by physicians. For the clinic sites, additional assessments included patient satisfaction and impact on health care costs.

Specific Aims:

  • Develop a system within three different ambulatory EHR systems in three different types of ambulatory settings that includes proactive followup of patients' response to treatment (including medication adherence and adverse events) and feedback to health care providers. (Achieved)
  • Assess the impact of automating the followup and feedback system. Impact will be measured in terms of: 1) diagnostic quality; 2) prevention of adverse events; 3) patient satisfaction with clinical care; and 4) health care costs. (Achieved)
  • Develop and evaluate an automated system for feedback to emergency medicine physicians of the concordance between their initial diagnoses and patients' final diagnostic outcomes. (Achieved)

2011 Activities: The 1-year no-cost extension provided the opportunity to complete data collection, analysis, and writing of results. Manuscripts under development during this period included a descriptive paper summarizing the results of the concordance analyses; lessons learned about implementing IVR for ambulatory followup; patient satisfaction results; and a main paper summarizing the whole study and outcomes, including clinical, costs, and physician and patient satisfaction. Presentations on project results were given, including a poster presentation focusing on the patient satisfaction survey at the American Medical Informatics Association spring meeting, and a second presentation on the development of the medication compliance scale presented at the Society for Behavioral Medicine. Dr. Berner also presented a Webinar about the project to researchers at Creighton University in March. As last self-reported in the AHRQ Research Reporting System, project progress was on track and project budget spending was on target. All project activities were completed when the project ended in August 2011.

Impact and Findings: Baseline data showed that 10-to-20 percent of ambulatory patients reported that their problems were not resolved within a week of their acute care visit. Many reported that their problem persisted after 3 weeks. A large proportion of patients did not contact their health care providers when they did not improve as expected. Patient satisfaction with the program was high throughout all phases of the program.

Physicians who viewed the feedback found it helpful. Cost analyses showed that if a followup system was implemented routinely the expense could be offset by increased revenue from return visits, with the potential to improve the quality of care and avert higher costs of hospitalizations. In the secondary study, the overall dissonance rate between ED and discharge diagnoses was approximately 10 percent. Providing feedback to physicians that could address the discordant diagnoses must address the workflow, confidentiality, and time constraints inherent in an ED setting.

Patients and providers appreciate "closing the feedback loop". Patients who receive followup calls are more satisfied with their overall care than those who do not. The project demonstrated that IVR systems are a feasible approach for patient followup in ambulatory settings. Costs for such followup can be offset by increased patient care revenue, and early followup may avert more costly health care expenses and can potentially improve the quality of care.

Target Population: Adults, Cerebral Palsy, HIV/AIDS

Strategic Goal: Develop and disseminate health IT evidence and evidence-based tools to improve health care decisionmaking through the use of integrated data and knowledge management. 

Business Goal: Synthesis and Dissemination