Impact of Health Information Technology on Clinical Care (California)

Project Final Report (PDF, 108.05 KB) Disclaimer

Impact of Health Information Technology on Clinical Care - 2008

Summary Highlights

  • Principal Investigator: 
  • Funding Mechanism: 
    RFA: HS04-012: Demonstrating the Value of Health Information Technology (THQIT)
  • Grant Number: 
    R01 HS 015280
  • Project Period: 
    09/04 – 09/08, Including No-Cost Extension
  • AHRQ Funding Amount: 
  • PDF Version: 
    (PDF, 51.37 KB)

Strategic Goal: To develop and disseminate health IT evidence and evidence-based tools to improve health care decisionmaking through the use of integrated data and knowledge management.

Business Goal: Knowledge Creation

Summary: This study evaluated the impact of health information technology (IT) on clinical care for patients with chronic diseases in a large, prepaid integrated delivery system (IDS), Kaiser Permanente-Northern California. This natural experiment involved the staggered introduction of a commercially available ambulatory health IT system with an electronic medical record, computer-based provider order entry, and embedded computer-based decision support systems across primary care teams serving over three million patients. Prior to this implementation, the health system introduced Web-based health IT tools in 2004 to assist with documentation, ordering, and consultations; these tools permitted clinicians to perform the basic clinical functions with minimal decision support and importantly, did not replace the paper medical record. In contrast, the new systems are intended to replace traditional records entirely.

Using a quasi-experimental design with concurrent controls, the project evaluated health IT-associated changes in quality, safety, and resource use in chronic disease care patients between 2004 and 2008. The research focused on the 780,000 patients with chronic diseases (asthma, congestive heart failure, coronary artery disease, diabetes mellitus, and hypertension) because these patients tend to require regular and complex ambulatory care, and may be sensitive to health IT effects. Measures of quality, safety, and resource use that are quantifiable both before and after the health IT intervention were used. To evaluate quality and safety, the association between health IT and guideline-consistent drug use and laboratory monitoring, drug adherence, and laboratory results were tested. These measures represent areas for which the IDS has clinical guidelines and consistent data records, yet significant room for improvement exists. These measures also may be affected indirectly by health IT, such as through the use of performance feedback or greater care coordination between providers. To evaluate resource use, the research assessed the association between health IT and office visits, emergency department visits, and hospitalizations.

Specific Aims

  • Design and implement provider and staff surveys. (Achieved)
  • Collect and analyze automated data. (Ongoing *)
  • Disseminate findings. (Ongoing *)

2008 Activities: Clinics continued to implement health IT systems throughout the year. Data collection and analysis continued. Preliminary findings suggest that a longer time frame for the study would be beneficial, as our data show a delay in the impacts of health IT systems after implementation.

Impact and Findings: To date, a number of preliminary examinations of the data have been conducted. Automated data (2004–2006) were used from outpatient visits to assess the association between the implementation of the new health IT system and clinical data quality, including the timeliness of data and the thoroughness of documentation for patient diagnoses in an integrated health system. The implementation of a new health IT system was found to be associated with a dramatic increase in the timeliness of diagnostic information from that system, but not in the number of diagnoses charted during patient visits. Preliminary analysis found no statistically significant association between health IT implementation and changes in physiologic outcomes for diabetics; however, there were substantial secular changes in low-density lipoprotein cholesterol levels. Further research is needed to assess the long-term effects; in analyses using longer followup periods, there appear to be statistically significant associations between health IT use and quality outcomes. In addition, various measures of health IT use have been examined. Based on these analyses, health IT presence alone (i.e., adoption or introduction date) is believed to not be a sensitive or useful predictor.

Using clinician survey data collected in 2005, the levels and patterns of clinician use of available health IT tools during primary care visits and the factors associated with systematic health IT use were examined. It was found that all clinicians reported at least some health IT use; however, the level of use varied significantly by function. In multivariate analyses, factors associated with systematic health IT use included perceived training adequacy, health IT incorporation into the clinical workflow, and hours worked per week. Clinician survey data collected in 2005 and 2006 were used to examine the impact of having an integrated health IT system on primary care clinician reports of the completeness and timeliness of all relevant clinical information, agreement on treatment goals and plans, and agreement on roles and responsibilities when multiple clinicians are involved with a patient’s care. Health IT introduction was found to be associated with substantial improvements in the timeliness and completeness of relevant clinical information and agreements on treatment goals and plans.

Selected Outputs

Bardach NS, Huang J, Brand R, et al. Evolving health information technology and the timely availability of visit diagnoses from ambulatory visits: a natural experiment in an integrated delivery system. BMC Med Inform Decis Mak 2009 Jul;9:35.

Grantee’s Most Recent Self-Reported Status (as of September 2008): This grant has been completed. While implementation of health IT was completed in 2008, data for all of the study covariates and outcomes will not be available until early 2009. In addition, it would be ideal to be able to incorporate as much additional followup time post-implementation as possible for many of the sites in order to capture any pattern in health IT-related outcomes after implementation. As of the conclusion of the grant, all milestones were either completed or on track, given the time frame across different primary care teams.

Milestones: Progress is mostly on track.

Budget: On target.

*Several aims of the grant were not completed prior to the scheduled conclusion of the grant (September 2008), yet, as other sources of funding have been secured, these aims are still targeted for completion.

Impact of Health Information Technology on Clinical Care - Final Report

Hsu, J. Impact of Health Information Technology on Clinical Care - Final Report. (Prepared by Kaiser Foundation Research Institute under Grant No. R01 HS015280). Rockville, MD: Agency for Healthcare Research and Quality, 2009. (PDF, 108.05 KB)

The findings and conclusions in this document are those of the author(s), who are responsible for its content, and do not necessarily represent the views of AHRQ. No statement in this report should be construed as an official position of AHRQ or of the U.S. Department of Health and Human Services.
Principal Investigator: 
Document Type: 
This project does not have any related event.
This project does not have any related resource.
This project does not have any related survey.
This project does not have any related project spotlight.
This project does not have any related survey.
This project does not have any related story.
This project does not have any related emerging lesson.


Health information technology (health IT) has great potential to improve health care; however, there is limited information about health IT effects on clinical care, especially with respect to commercially available technology in community-based ambulatory clinics. The dearth of quantitative information is particularly concerning given the significant resources needed for implementation and the limited adoption of health IT in the United States. We evaluated the effects of ambulatory health IT on quality, safety, and resource use between 2004 and 2007. We used a quasi-experimental pre-post design with concurrent controls, within a large, integrated health delivery system (IDS). In this natural experiment, the staggered health IT implementation across 110 primary care teams occurred over 38 months, starting in November 2004. This new commercial health IT included an electronic medical record integrated with an order-entry system and decision-support; the previous system was primarily based on the paper medical record. The IDS's previous automated databases permitted consistent capture of our outcomes and patient-level covariates before and after the new health IT, but are notably distinct from the new health IT with respect to providing real-time clinical data. We focused on a cohort of 780,000 IDS members with at least one of five chronic diseases (asthma, coronary artery disease, heart failure, diabetes mellitus, and hypertension) in January 2004; these patients may be particularly sensitive to health IT-related changes in ambulatory care. The presence of health IT in the primary care team was the main predictor. The quality and safety measures included guideline-adherent drug dispensations and laboratory monitoring, drug adherence, and physiologic disease control (as measured by laboratory tests). The resource-use measures include emergency department visits, nonelective hospitalizations, and office visits. Using Poisson regression models with patient-level random effects, we tested the hypotheses that health IT is associated with improved quality and safety measures, and health IT is associated with lower visit rates. We had the ability to detect small changes in our outcomes, e.g., 80 percent power to detect a 3 percent change in the relative rate of nonelective hospitalizations in the CHF group. We made adjustments for patient, insurance, and organizational factors, including socioeconomic status, case mix, cost-sharing, and care-delivery structure, using automated data and annual surveys. In short, this natural experiment provided an ideal setting to understand the value of health IT in community-based, ambulatory care.