Project Details -

Ongoing

Traditional emergency department (ED) triage tools, which include a survey index still in use by approximately 70 percent of hospitals nationwide, are dated and highly subjective. Triage, the sorting of patients to determine when, where, and by whom they will be seen, is prone to errors when implemented in the ED, both in under- and over-triaging. This mis-triage can worsen crowding and delays in care, decrease quality of care, and increase disparities. Existing predictive models outperform more traditional triage tools; however, they have significant flaws: a focus on hospital admissions or critical care, even though more than 80 percent of ED visits do not result in an admission; a failure to include pediatric patients, although this population represents 25 percent of ED visits; a failure to consider health equity in design or evaluation of prediction models; and a failure to prospectively validate these tools in real-time use by studying their impacts on key patient safety and quality measures.

This research will design, implement, and evaluate an ED triage clinical decision support (CDS) tool integrated into an electronic health record (EHR) that uses machine learning algorithms to improve triage.

The specific aims of the research are as follows:

Refine triage models to predict critical illness, hospital admission, and fast-track eligibility separately among adult and pediatric patients, and measure algorithm biases.
Design, build, and evaluate a clinician-facing triage CDS.
Assess the impact of the CDS in a randomized trial.

To prepare for this work, the researchers previously completed significant preliminary analyses, including preparing a large study cohort consisting of over six million adult and pediatric ED encounters across 21 EDs, assessing current limitations of ED triage in the study setting, and developing machine learning models that predict both patient acuity and resource needs at triage.

The current triage prediction model will be refined using machine learning algorithms trained on the large study cohort to improve prediction of critical illness, hospital admission, and fast-track eligibility. Incorporating user-centered design methodology, the clinician-facing CDS tool will be developed, built, and integrated into the EHR with significant stakeholder and end-user engagement. It will display personalized risk predictions for each of the outcomes and a final triage assignment recommendation based on these risk predictions as part of triage workflows. The tool will be implemented across 21 ED sites, with significantly different patient populations--including both adults and children--access to followup care, ED volumes, and availability of resources. The researchers will then evaluate the impact of the CDS in a randomized trial on the primary outcomes of timeliness of care for critically ill patients, appropriate early identification of fast-track eligible patients, and ED length of stay. As part of the research, biases of the algorithms will be measured by assessing bias by race, gender, and socioeconomic status among the primary outcomes, with improvements made in the model to improve equity in model predictions. The researchers will use FHIR standards so the CDS tool can be shareable and interoperable.

The researchers hypothesize that the use of deep learning models to inform an ED triage tool integrated into the EHR will improve the prediction of patient acuity and complexity, thus leading to safer, higher quality, and more equitable care.

Machine Learning to Improve Patient Triage in the Emergency Department

Theme:

Supporting Health Systems in Advancing Care Delivery

Subtheme:

Advancing Health Equity

The use of an emergency department triage tool informed by machine learning has the potential to improve predictions around patient health severity, leading to safer, higher quality, and more equitable care.

Current triage processes are lacking in the ED, compromising patient care

When patients arrive at the emergency department (ED), they are assigned a triage category that impacts their downstream care, including when they will be seen (immediately versus delayed), what type of room they will need (high or low acuity), and what type of provider will see them (e.g., physician, nurse practitioner, or physician assistant).

Traditional tools currently used by ED triage nurses are dated and highly subjective, which can lead to errors in under- and over-triaging. A mis-triage can worsen patient crowding in the ED and delay care for patients who are critically ill and require immediate intervention, such as those suffering from a stroke or heart attack.

Leveraging the power of machine learning can improve triage

While electronic health records (EHRs) do contain comprehensive information about patients, ED triage nurses do not always have the time and processes in place to assess all relevant information to support better triage. Current patient-specific information like comorbidities, medications, housing status, or previous hospitalizations can all inform whether a patient is in higher need of care.

Dr. Dana Sax, a practicing emergency medicine physician, and a team of researchers at Kaiser Permanente Northern California (KPNC) are currently working to design, implement, and evaluate an ED triage clinical decision support (CDS) tool integrated into the EHR that uses machine learning to improve triage.

“We have rich data available in our EHR that can be used, in combination with presenting complaint and vital signs, to assist nurses with predicting patient acuity and resources needs in the emergency department. We can use these data to build and test prediction models targeting key outcomes, such as critical illness or hospital admission. Once refined and built into the EHR, these triage prediction models can be used in real time to help inform accurate sorting of patients on arrival. Carefully designed decision support that integrates risk estimates has the potential to improve the quality, safety, and equity of the care that we provide in the emergency department.” — Dr. Dana Sax

Improved triage can improve the quality of patient care in the ED

Using extensive EHR data from over six million ED encounters across KPNC medical centers, the team is developing and refining machine learning–based risk stratification models that predict critical illness, hospital admission, and fast-track eligibility, meaning the ability to separate low severity patients from other patients in the ED for fast, nonacute treatment and rapid discharge.

Refinements to the models will also take into account pediatric patient data, since approximately one quarter of ED visits are for children. Equity is also important to address. In previous research, the team found that Black patients had 20 percent greater odds of mis-triage compared with White patients, and found additional disparities by income and gender, highlighting an urgent need to improve triage equity. To address this, they will assess potential biases in predictor variables and will prioritize models with the best performance and least biased predictors. The goal is to use these data to improve the algorithms to improve health equity in model predictions.

Using a human factors framework by involving nurses and physicians in the testing and development, the clinician-facing CDS tool will be developed, built, and integrated into the EHR. The study team expects the digital health tool will display personalized risk predictions for each of the outcomes, although how this information is presented and best integrated into triage workflows will be informed by end-user feedback as part of this study.

The tool will be implemented across 21 KPNC ED sites that have significantly different patient populations (including both adults and children), access to followup care, ED volumes, and availability of resources. Dr. Sax and the team will evaluate the impact of the tool on the timeliness of care for critically ill patients, appropriate early identification of fast-track eligible patients, and ED length of stay. This ED triage tool integrated into the EHR should improve the prediction of patient illness severity and complexity, thus leading to safer, higher quality, and more equitable care.