Improving Safety and Quality with Outpatient Order Entry - 2008

Principal Investigator
Funding Mechanism
RFA: HS04-012: Demonstrating the Value of Health Information Technology (THQIT)
Grant Number
R01 HS 015226
Project Period
09/04 – 08/08, Including No-Cost Extension
AHRQ Funding Amount
Summary Status as of
August 2008, Conclusion of Grant

Strategic Goal: Develop and disseminate health IT evidence and evidence-based tools to improve the quality and safety of medication management via the integration and utilization of medication management systems and technologies.

Business Goal: Knowledge Creation

Summary: The United States health care system continues to face enormous quality, safety, and cost challenges, many of which occur in the ambulatory setting. Quality gaps that are particularly relevant in this setting include a high incidence of adverse drug events and lack of compliance to established guidelines for preventive care, chronic disease management, and test result followup. Two forms of health information technology (IT), clinical decision support systems (CDSS) and ambulatory computerized physician order entry (ACPOE), have been touted as powerful and sustainable interventions to address these quality concerns. Prior work suggests that CDSS work best when they are speedy and well-integrated into clinicians’ workflow. Therefore, tight integration of CDSS with ACPOE is a promising strategy for improving quality and efficiency in the ambulatory setting. When ACPOE is linked with advanced CDSS, clinicians can be prompted at various points during their workflow about the desirable course of action and simultaneously be given the opportunity to execute the action (by ordering it) with minimal effort. It is hypothesized that the value of ACPOE integrated with advanced CDSS lies not only in improved medication safety and guideline compliance, but also in improved efficiencies for the individual provider and the health care system. Further, the overall value added by these systems is hypothesized to outweigh their costs.

In this project, study physicians worked with the Information Systems staff to develop the clinical content and to convert this content into interpretable clinical decision support rules within the ACPOE system. During the initial study phase, four outpatient primary care clinics, affiliated with Partners HealthCare System and utilizing the electronic health record (EHR) for at least 24 months, implemented ACPOE. The implementation phase took between 8 to 12 weeks. The ACPOE system has undergone nine software releases to add functionality and improve its user experience. As of August 2008, seven practices are using the system. Over 145,000 orders have been generated across approximately 109,000 encounters. Implementation of an EHR-integrated with ACPOE and CDSS also occurred at three health centers affiliated with another institution, Atrius Health. EHR implementation occurred in phases, taking several months to reach full functionality.

Specific Aims

  • Evaluate physicians’ use of the EMR system in chronic disease management, preventive care, and medication monitoring in a controlled trial for Actionable Reminders, Advanced Results Manager, and Order Tracking modules. (Ongoing*)
  • Perform quantitative analyses of the impact of ACPOE system on clinicians’ time utilization and frequency of unnecessary redundant testing. (Ongoing*)
  • Perform qualitative analyses of users’ satisfaction with system, including both pre-post surveys and longitudinal studies. (Achieved)
  • Apply cost-benefit analysis to ACPOE implementation. (Upcoming*)

2008 Activities: The Actionable Reminder Intervention was completed on March 31, 2008. Analysis will be completed by January 2009, and an abstract will be submitted for presentation at the Society for General Internal Medicine Annual Meeting in May 2009. The functional and technical specifications for the Advanced Results Manager module of the ACPOE system were completed in early 2008. However, due to a major enterprise architecture initiative, development was on hold until November 2008. Coding was estimated at 6 to 8 weeks, and the functionality will go live with the next major EHR release in spring 2009.

Impact and Findings: Before ACPOE implementation, 15 clinicians were observed, treating 193 patients; after ACPOE implementation, 15 clinicians were observed, treating 137 patients. Twelve of these clinicians were observed both before and after implementation. The data suggest that the ACPOE implementation did not significantly impact the duration of time spent in clinic and time spent face-to-face with patients. Compared to pre-ACPOE, the distribution of time spent in major activities also did not significantly change after ACPOE. Further analysis is under way to examine the impact on specific clinician clinic activities that occur between patient visits.

The response rate for the baseline survey was 55 percent (144/261). The survey revealed that 52 percent of primary care physicians (PCPs) did not have a system for tracking test results, 62 percent did not have a system to detect a missed ordered test, and 32 percent were not satisfied with how they managed test results. ACPOE addresses common physician concerns, such as test tracking and notification of missed tests. However, physicians were concerned about the impact of ACPOE on workflow and the value of its features, and these concerns will need to be addressed to ensure acceptance.

Results for the longitudinal study indicate that over the course of a year, the proportion of clinicians agreeing that the EHR improved quality of care increased. The proportion of clinicians who agreed that the EHR reduced medication-related errors and improved followup of test results also increased. An increasing proportion of respondents agreed that communication among clinicians improved, and a decreasing proportion reported a worsening in the quality of patient-physician interactions.

While clinicians may perceive some initial problems with a new EHR, they become significantly more receptive to it within 1 year of implementation. The research team is currently performing an analysis to determine the baseline rates of redundant testing and the potential cost savings that could be achieved if redundant testing decision support rules were incorporated into order entry. While this decision support was not implemented during the grant, it is anticipated that the findings will inform the institution and others whether to use such decision support. There is some variation in the perceived impact on efficiency.

Selected Outputs

El-Kareh R, Gandhi TK, Poon EG, et al. Trends in primary care clinician perceptions of a new electronic health record. J Gen Intern Med. 2009;24(4):464-8.

El-Kareh R, Gandhi TK, Poon EG, et al. Trends in clinician perceptions of a new electronic health record. AMIA Annu Symp Proc 2008 Nov 6:940.

Ungar JP, Gandhi TK, Poon EG, et al. Impact of ambulatory computerized physician order entry on clinicians’ time. AMIA Annu Symp Proc 2008 Nov 6:1158.

Gandhi TK, Poon EG, Sequist TD, et al. Primary care clinician attitudes towards ambulatory computerized physician order entry. AMIA Annu Symp Proc 2005:961.

Grantee’s Most Recent Self-Reported Quarterly Status (as of August 2008): This grant is completed. Implementation for several of the planned modules, including some CDSS, was delayed beyond the ending period of the grant. Research, knowledge transformation, and coding efforts were largely completed, and based on data from implementation of ACPOE.

Milestones: Progress is mostly on track.

Budget: On target.

* Several aims of the grant were not completed prior to the scheduled conclusion of the grant (August 2008), yet, as other sources of funding have been secured, these aims are still targeted for completion.