Use of Electronic Referral System to Improve the Outpatient Primary Care–Specialty Care Interface - 2009

Principal Investigator
Contract Number
Project Period
09/07 – 09/09
AHRQ Funding Amount
Summary Status as of
September 2009, Completion of Contract

Strategic Goal: Develop and disseminate health IT evidence and evidence-based tools to support patient-centered care, the coordination of care across transitions in care settings, and the use of electronic exchange of health information to improve quality of care.

Business Goal: Implementation and Use

Target Population: General

Summary: Poor communication and coordination of care between primary care and specialty care providers leads to major inefficiencies in health care delivery. In resource-constrained settings, these inefficiencies exacerbate mismatches between the supply and demand for specialist services. This project evaluated the implementation of a Web-based electronic referral system (eReferral) developed by the University of California San Francisco (UCSF) and San Francisco General Hospital (SFGH). The eReferral system is staffed by specialist reviewers to allow clarification of the consultative question, requests for additional evaluation, and triaging of appointment requests. The study included a multimethod evaluation of a two-part intervention of extending the use of eReferral and making improvements designed to support primary care providers’ (PCPs’) use of the system.

The evaluation of the eReferral system consisted of three components:

  • Secondary analyses of quantitative data from SFGH administrative systems, eReferral system usage logs, and data from two quality improvement surveys previously conducted among SFGH providers comparing indicators of accessibility, efficiency, and quality of specialty care before and after the use of eReferral.
  • Semi-structured interviews with eReferral users in primary care and in specialty clinics to better understand users’ views about the benefits and drawbacks of eReferral and to identify best practices in implementing the system.
  • Simulation modeling to document the business case for implementing eReferral for specialty and primary care sites and to project the system’s implications for health care costs and utilization of services.

The evaluation of this system centered on changes in the quality, efficiency, accessibility, and patient-centeredness of outpatient specialty care. Both specialty and referring physician users viewed eReferral as a success, reported improved communication between each other, and increased access to specialty care. They also perceived any increases in the time needed for the eReferral process as valuable contributions to patient care; however, further validation may be needed to ensure the work process models adequately reflect the time and cost tradeoffs. Establishing valid simulation models that can predict the costs and benefits of electronic referral system designs will be important for creating successful electronic referral systems in other settings of care.

Specific Aims

  • Compare changes among specialty clinics on indicators of the quality, efficiency, accessibility, and patient-centeredness of outpatient specialty care before and after use of eReferral. (Achieved)
  • Assess distinctive eReferral implementation practices among specialty and primary care sites and explore how these practices might influence the system’s success or failure in achieving business and health care goals. (Achieved)
  • Estimate the net costs (versus savings) of implementing eReferral for specialty and primary care sites and document the business case for the system’s adoption and use. (Achieved)

2009 Activities: The focus of activity was on defining the primary care side intervention and making improvements to the eReferral user interface, including the following additions.

  • Added a Scheduling Considerations Box: This new function allows referring providers to let clinic reviewers/schedulers know about scheduling constraints the patient may have (e.g., is out of the country during the summer, can only come for afternoon appointments, etc.) with the goal of improving show rates.
  • Added a Worklist Reorganization/Display: This new function allows for the following worklists to be accessible: 1) referred patients worklist, 2) a PCP’s own worklist, 3) other providers’ worklists, 4) referring location worklist, and 5) primary care clinic’s worklist.
  • Added a Nonclinical Note: This new function allows the user to enter notes about logistical/scheduling/contact issues that gets attached to the eReferral—used most by primary care clinics that are tracking their referrals.
  • Added the ability to remove eReferrals (and restore them) from provider worklists.
  • Added the ability to save a draft eReferral for completion within 14 days.

Additional activities completed throughout the year included the drafting of the UCSF-SFGH eReferral Implementation Handbook and the completion of post-intervention interviews. The project team completed a total of 28 interviews, which provided rich data on the implementation and outcomes of the intervention. When it became evident that the team was approaching theoretical saturation, the team shifted resources from interviewing to focusing on qualitative coding and analysis of the interview transcripts.

Throughout the implementation, the development team was responsive in addressing system issues as well as organizational issues that arose, such as onerous requirements for scheduling appointments that were imposed by one specialty clinic, and they conducted a relatively slow pace of rollout across the available specialty clinics. The project team also worked on a manuscript reporting results from the SFGH specialist survey and made plans for three additional manuscripts.

Impact and Findings: Analysis of the eReferral system logs demonstrated substantial initial decreases in wait times for routine new patient appointments for seven of eight medical specialty clinics. The changes in wait times resulted from increased appointment availability due to appointments “not initially scheduled” so that initial workup could be completed by PCPs and from referrals that never resulted in an appointment with PCP advice being delivered through eReferral instead. The eReferral system also enabled acceleration of more urgent care, indicated by an estimated 37 percent increase in expedited referrals.

Survey results showed that specialists reported significant improvements in their ability to identify the consultative question and in appropriateness of referrals. PCPs reported that eReferral improved quality of care for their patients but that information technology connectivity posed significant problems for some clinics.

User interviews revealed that most PCPs and specialists were satisfied or very satisfied with eReferral, despite a variety of challenges. A major driver of the system’s acceptance was the perception that the system substantially improved access to specialty care, quality of care, and administrative efficiency in submitting and managing referral requests. Numerous interview participants reported that by using a specialist reviewer to review and triage referral requests, eReferral prevents premature and inappropriate referrals, for example those where the patient should have further diagnostic testing before being seen by the specialist or where the patient should be referred to a different specialty service. This was viewed as a major benefit by specialists but was also seen positively by referring providers. These benefits were mediated largely by improved communication between primary care and specialty care providers. Uptake may have been enhanced by factors including mandatory use of the system (no paper alternative), the user-interface, which users perceived as intuitive and easy to learn, and process adaptations implemented by some practices.

Simulation of typical referral work processes was built for average referral volumes in the medical and surgical departments of SFGH (selected departments, 854 and 1,212 annual referral requests, respectively). Simulation results predicted that the system would reduce the number of specialist appointments needed to care for the fixed referral base by 29 percent for medical specialties and 33 percent for surgical specialties. To achieve this access gain in the medical specialties, a specialist reviewer spent an estimated 9.4 minutes per referral request (133.9 hours for 854 reviews). The net amount of time needed for eReferral reviews exceeded the 29 percent estimated reduction in specialist time spent on visits (75.6 hours per 854 referrals), and resulted in 9.5 percent additional referral processing labor costs. However, in surgical clinics, eReferral reviews were conducted by lower-cost nurse practitioners. Thus, the reviewer time needed (8.1 minutes per referral, 163.9 hours for 1,212 referrals) costs substantially less than the surgeon time saved on visits (100.7 hours for 1,212 referrals), yielding a 22.5 percent cost reduction using eReferral. PCPs spent 2.7 more time submitting, responding, and revising referrals compared to the system of paper-based referral requests. There were substantial savings in staff time in both specialty clinics and PCP offices. Overall, the labor costs were projected to be modestly higher for eReferral to medical subspecialty clinics and lower for eReferral to surgical clinics.

Selected Outputs

The study yielded an implementation handbook, cost tool, and summaries of findings for dissemination to other care settings.

The final report is forthcoming in 2010.