ML-ROVER: Machine Learning to Reduce Laboratory Test Overutilization

Overutilization of laboratory testing poses significant risks, particularly in pediatric critical care, where frequent testing can lead to complications such as iatrogenic anemia, infections, venipuncture injuries, organ dysfunction, and increased mortality. Vulnerable populations, such as children with cyanotic heart disease or sepsis, face heightened risks, with phlebotomy accounting for 73 percent of daily blood loss in some pediatric intensive care units (PICUs). Despite recommendations from medical societies to reduce unnecessary testing, interventions have largely failed to produce sustainable improvements, and alert fatigue remains a serious issue—over 95 percent of clinical decision support (CDS) alerts are overridden by clinicians, contributing to burnout and potential patient harm.

While machine learning (ML) offers a promising path for reducing laboratory overuse by enhancing CDS tools, most implementations lack the usability and integration required for real-world impact. Many CDS systems are deployed without clinician input or post-implementation evaluation, resulting in burdensome, often ignored alerts. Though some ML models for reducing testing show potential in adult care, pediatric-focused solutions are scarce. This highlights the urgent need for adaptable, data-driven CDS tools that fit clinical workflows, reduce disparities across healthcare settings, and support better outcomes in pediatric critical care. A thoughtful approach focused on user-centered design and practical integration is essential to achieving sustainable change.

The specific aims of the research are as follows:

• Develop and validate ML models to predict laboratory test results using an existing multicenter electronic health record (EHR) database of critically ill children.
• Assess the Practical Robust Implementation and Sustainability Model (PRISM) contextual factors to inform the design of a CDS tool for laboratory test use that is locally relevant and scalable.
• Extend, deploy, prospectively evaluate, and calibrate the ML models.
• Develop an EHR-embedded CDS tool and assess its usability by applying the principles of user-centered design.

The researchers will take a multi-phase approach, leveraging the PICU Data Collaborative (PDC) to develop, validate, and implement a ML-based CDS tool aimed at reducing unnecessary lab testing in PICUs. They will train ML models on the PDC’s multicenter EHR dataset to predict lab results, targeting over 90 percent accuracy. To ensure effective implementation, they will use the PRISM framework to assess contextual factors and identify barriers and facilitators at different PICU sites, guiding the development of scalable, site-specific tools. The dataset will also be expanded to include critical variables such as fluid intake, vascular access, and transfusion data. Prospective evaluations at two sites will confirm the model’s accuracy and assess its impact on reducing lab tests. Finally, the CDS tool will be integrated into the EHR using user-centered design principles, with usability testing ensuring its practicality and effectiveness. A three-month pilot will assess adoption, laying the groundwork for future multicenter trials.

The researchers' strategy combines advanced ML techniques with clinical insights to create a sustainable solution for reducing laboratory test overutilization in pediatric critical care. They aim to establish a scalable process for developing ML-based CDS tools that not only addresses lab overutilization but also serves as a foundation for broader clinical challenges. Successful completion will enable future trials to assess the tool's effectiveness across multiple centers, guide the creation of additional risk prediction models, and support rapid innovation with new data and sites. Their approach emphasizes sustainability and flexibility, ensuring the ongoing validity and relevance of the tools in tackling high-value clinical issues beyond laboratory testing.