Surveillance for Adverse Drug Events in Ambulatory Pediatrics - 2010

Target Population: Cancer, Chronic Care*, Cystic Fibrosis, Pediatric*, Sickle Cell Anemia

Summary: Adverse drug events (ADEs) comprise one of the largest categories of adverse events in the principal studies examining the epidemiology of patient safety. Measurement of ADEs was identified as a critical patient safety metric in the Institute of Medicine’s 2004 report on patient safety and in their 2001 National Healthcare Quality Report. Measuring the incidence of ADEs in care environments is essential to: 1) establish a baseline performance metric to measure improvement; 2) separate medication errors and system failures that result in harm to patients from those that do not; and 3) accurately direct interventions to prevent the failures that are harmful. Despite extensive literature on medication safety, medication errors, and adverse drug events in adult populations, little is known about the frequency and nature of these events in children and less is known about ADE incidence in children with chronic disease.

This project uses automated surveillance to measure the incidence of outpatient ADEs suffered by children with sickle cell disease, cystic fibrosis, or cancer, either in the emergency department (ED) or during the transitions between hospital admission and discharge. The project will analyze data generated from BJC HealthCare system, which includes the St. Louis Children’s Hospital. The St. Louis Children’s Hospital ED uses the Wellsoft ED computer system and the McKesson Corporation’s Certification Commission for Health Information Technology-certified Horizon Expert Documentation for inpatient care. The BJC/Washington University Medical Informatics Laboratory group uses its rule-based expert systems architecture for discrete data, and the open-source natural language processing system, cancer text informatics extraction system for textual data, to automatically scan laboratory, pharmacy, demographic, documentation, and diagnostic code data from the target populations for “signals,” or data combinations that suggest the occurrence of an ADE. This automated system is being evaluated for efficiency (positive predictive value [PPV] and time and resource efficiency) and effectiveness in ADE detection compared with targeted explicit chart review. The project will also examine the impact of access to ADE metrics by practitioners. The data from the system will be used to improve strategies for medication use safety in clinic, emergency, and inpatient environments.

Specific Aims:

• Implement an automated surveillance system for measuring the incidence of ADEs occurring in the outpatient setting (including the emergency department) in pediatric patients with specific chronic diseases that result in the need for emergency department care or admission to the St. Louis Children’s Hospital. (Achieved)
• Use the automated surveillance system for measuring the incidence of ADEs occurring in these patient populations during transitions in care from outpatient to inpatient setting, e.g., originating during the admission process. (Ongoing**)
• Use the automated surveillance system to measure the incidence of ADEs in the target pediatric populations within four weeks of discharge. (Ongoing**)
• Evaluate the performance of the event detection system as employed in the three above aims. (Ongoing**)

2010 Activities: The chart review was initiated during this period with a sample population size of 394 patients randomly selected from 1,990 total study patients. The sample was stratified by patient condition: sickle cell, cystic fibrosis, or cancer. A study pharmacist reviewed the chart review population for ADEs using a chart review tool designed with prompts for detecting ADEs. For each prompt, the pharmacist assessed if an ADE occurred and documented the ADE. Another study pharmacist matched chart review ADEs to ADEs detected by automated alerts. A study physician reviewed every chart review event that was not found by the automated method, or was found by the automated method but there was a difference in the assessment of whether an ADE was present.

Study pharmacists re-classified the automated system alerts into events. Events were defined as a single alert or collection of alerts that the reviewer believed to be an ADE or potential ADE. The study team defined the standard for the comparison as the combination of ADEs found by both the automated system and chart review, as determined by expert review. Using this standard, they will produce three different comparisons.

1. Comparison of chart review events alone with the standard.
2. Comparison of the pharmacist assessment of alerts generated by the automated system with the standard.
3. Comparison of the automated system without pharmacist review of the alerts with the standard.

The second and third comparisons highlight the difference between the number of detected true events and the number of alerts to detect these events. Each comparison will include sensitivity and PPV. Negative predictive value and specificity will not be established as the number of events that were not found by either chart review or the automated system is unknowable.

Grantee’s Most Recent Self-Reported Quarterly Status (as of August 2010): Project progress is on track. Project spending was significantly under budget due to delays in the project and transition to a new principal investigator.

Preliminary Impact and Findings: In the study population of 1,990 patients, there were 212 discrete alerts that detected 73 ADEs, and 1,055 natural language processing (NLP) alerts that detected 70 ADEs. Preliminary analysis shows the system is discovering harmful ADEs in 1.3 percent of inpatient admissions with a PPV of 13 percent. Data suggest that the rate of NLP-detected ADEs per year (i.e., alone, not counting ADEs detectable with discrete alerts) in the study populations was approximately 6.8 events per 100 patients per year; this is the same order of magnitude described by studies in adult populations.

Strategic Goal: Develop and disseminate health IT evidence and evidence-based tools to improve health care decisionmaking through the use of integrated data and knowledge management.

Business Goal: Knowledge Creation

*AHRQ Priority Population.

**Several aims were not completed prior to the scheduled conclusion of the grant period, however, work continues on the aims.