Integration of an NLP-based Application to Support Medication Management (Massachusetts)

Project Final Report (PDF, 1.47 MB) Disclaimer

Project Details - Ended

Project Categories

Summary:

Medication reconciliation (MedRec), the process of comparing a patient's medication order with all the medications the patient has been taking, is critical to preventing medication errors. However, it is challenging because patients see many clinicians who don’t share a common medication list. In addition, medication information frequently exists in free-text clinical notes and is not easily available for review.

This project builds on a previous AHRQ-funded project, Improving Outpatient Medication Lists Using Temporal Reasoning and Clinical Texts, that developed a natural language processing (NLP) system called the Medical Text Extraction, Reasoning, and Mapping System (MTERMS). MTERMS extracts and encodes medication information from electronic clinical notes into a structured format. That project showed that in patients with chronic diseases, 31 percent of the active medications mentioned in providers’ notes were missing from their medication lists. In this followup project, the investigators developed and evaluated an NLP-based tool to support medication list management at the point of care.

The specific aims of this project were as follows:

  • Identify the requirements, use cases, workflow issues, and barriers to and facilitators of using clinical notes and NLP in the MedRec process using focus groups with providers and domain experts. 
  • Design generic system architecture and an application that integrates an NLP system and a Web-based user interface within an existing MedRec system. 
  • Pilot the tool in two primary care clinics and measure the utilization, usability, performance, and feasibility of the proposed methods and the tool in improving the process of MedRec in the outpatient setting using both quantitative and qualitative methods. 
  • Distribute the methods and the tool so they are widely available to other researchers and health care institutions for non-commercial use. 

The investigators conducted a qualitative analysis to understand user requirements, use cases, system functional specifications, workflow issues, and barriers to and facilitators of using clinical notes for MedRec in the ambulatory setting. This information was used to design, develop, and implement a real-time Web-based tool called NotesLink, a Web application built upon the MTERMS NLP system. NotesLink presents NLP output and links to original notes to facilitate clinicians' review of medication discrepancies between notes and the medication list.

The investigators conducted a pilot study at two Brigham and Women’s Primary Care practices, measuring the performance and feasibility of the NotesLink system in improving MedRec. To evaluate the system, the investigators randomly selected two samples of patients. The first group was used for a comprehensive evaluation of NotesLink; the second group had at least one medication discrepancy identified by NotesLink and was used to evaluate the accuracy of the system. Results generated by the system were compared to a gold standard created by domain experts. Preliminary findings have shown that of 1,098 medication mentions found in 163 notes representing 306 unique medications, NotesLink achieved a precision of 70.1 percent and a recall of 73.7 percent in identifying all medication discrepancies and status. Fifty-two medication discrepancies were identified, of which 50 percent represented possible true discrepancies.

This work identified gaps and challenges of using advanced information technologies combining NLP and automated decision support to facilitate MedRec at the point of care and will inform future development of comprehensive medication reconciliation systems.

Integration of an NLP-based Application to Support Medication Management - 2012

Summary Highlights

  • Principal Investigator: 
  • Funding Mechanism: 
    PAR: HS08-269: Exploratory and Developmental Grant to Improve Health Care Quality through Health Information Technology (IT) (R21)
  • Grant Number: 
    R21 HS 021544
  • Project Period: 
    July 2012 – June 2014
  • AHRQ Funding Amount: 
    $297,565
  • PDF Version: 
    (PDF, 318.18 KB)

Summary: Having an accurate and complete medication history at the point of care is crucial to the delivery of high-quality care and prevention of prescribing and medication administration errors. To meet Stage 1 of the Centers for Medicare & Medicaid Services Electronic Health Records (EHR) Incentive Program Meaningful Use requirements, EHRs must be able to provide users with the ability to perform medication reconciliation—the process of comparing a patient’s medication order to all of the medications that the patient has been taking. While much work has been done to facilitate the medication reconciliation process at patient care transitions, most of it has taken place in inpatient settings. Medication reconciliation in the ambulatory setting is challenging because clinicians may be unaware of medications prescribed by other providers. Additionally, the medication reconciliation technology in use today provides support using data from the structured fields in an EHR, while critical medication information may exist in an unstructured format in the EHR’s free-text clinical notes.

This project builds upon a previous AHRQ-funded project, Improving Outpatient Medication Lists Using Temporal Reasoning and Clinical Texts, for which the project team developed a natural-language processing (NLP) system called the Medical Text Extraction, Reasoning and Mapping System (MTERMS) to extract and encode medication information from electronic clinical notes in a structured format. This study showed that 31 percent of the active medications mentioned in providers’ notes were missing from the EHR’s structured medication lists of patients. In addition, this study found that providers often needed information beyond the medication list to make clinical judgments, changes, and other decisions. The overall goal of this followup study is to use NLP and other technologies to develop novel ways to facilitate the medication reconciliation process in the ambulatory setting. The team hypothesizes that the approach and the system based on NLP and information retrieval tools will help providers conduct medication reconciliation and therefore improve the accuracy and completeness of medication lists.

Specific Aims:

  • Identify the requirements, use cases, workflow issues, and barriers to and facilitators of using clinical notes and NLP in the medication reconciliation process. (Achieved)
  • Design a generic system architecture and an application that integrates an NLP system and a Web-based user interface within an ambulatory EHR system. (Ongoing)
  • Pilot this study in two primary care clinics and measure the utilization, usability, performance, and feasibility of the proposed methods and the tool in improving the process of medication reconciliation in the outpatient setting. (Upcoming)
  • Distribute methods and the tool so they are widely available to other researchers and health care institutions for non-commercial use. (Upcoming)

2012 Activities: Dr. Zhou and her research team conducted two focus groups with health care providers and one focus group with technical staff to identify the requirements, use cases, workflow issues, and barriers to and facilitators of using clinical notes and NLP in the medication reconciliation process. Health care providers who participated in the focus groups were from the two primary care sites where the NLP- based medication reconciliation tool will be implemented. The goal of the focus groups was to establish the current processes that various clinicians, including doctors, pharmacists, nurses, and physician assistants, undertake when conducting medication reconciliation. The providers agreed that the tool will help them identify missing medications found only in clinical notes and will aid them in updating patient medication list.

Through the focus groups, the research team also realized that providers often have to deal with extensive medication lists, which sometimes include discontinued medications that have not been retired. Health care providers would therefore find value in a tool that can help them identify discontinued medications from the medication list. The focus groups have also provided the team with insight into the preferred look of the user interface. Some of the suggestions have included highlighting the signifiers from which the system has made its references regarding a medication discrepancy, and then not only providing the link to the note, but also highlighting the extracted phrases within that note.

The technical focus group was conducted in November. The goal was to establish the requirements necessary for the tool’s implementation and to understand participants’ experience with medication reconciliation technology. A number of key issues, including the consideration of patients with multiple physician visits in one day, were discussed. The active medication lists for such patients will need to be loaded into the NLP program multiple times during the day to account for changes made during each visit. Currently, the NLP program is set up to analyze patient profiles once daily. Another issue dealt with multiple medication mentions within the same note. Manual review revealed that medications listed at the beginning of a provider note were not necessarily the medications with which the patient left at the end of the visit. This issue will be addressed by tagging section headings within each note and medication changes noted in the plan section of the note will be given priority. Additionally, rules deciphering the current status of the medication and whether it is being continued, discontinued, or newly started will also be implemented.

The preliminary Web page design for the medication reconciliation application has also been developed and will include sections for missing and discontinued medications, as well as an informational section referencing all medications mentioned within a patient’s note history. The exact layout of these headings will be decided upon after a few interface variations are tested and voted on by the project team. As last self-reported in the AHRQ Research Reporting System, project progress and activities are completely on track as is the project’s budget spending.

Preliminary Impact and Findings: This project has no findings to date.

Target Population: General

Strategic Goal: Develop and disseminate health IT evidence and evidence-based tools to improve the quality and safety of medication management via the integration and utilization of medication management systems and technologies.

Business Goal: Knowledge Creation

Integration of an NLP-based Application to Support Medication Management - Final Report

Citation:
Zhou L. Integration of an NLP-based Application to Support Medication Management - Final Report. (Prepared by Brigham and Women's Hospital under Grant No. R21 HS021544). Rockville, MD: Agency for Healthcare Research and Quality, 2014. (PDF, 1.47 MB)

The findings and conclusions in this document are those of the author(s), who are responsible for its content, and do not necessarily represent the views of AHRQ. No statement in this report should be construed as an official position of AHRQ or of the U.S. Department of Health and Human Services. 
Principal Investigator: 
Document Type: 
This project does not have any related resource.
This project does not have any related survey.
This project does not have any related project spotlight.
This project does not have any related survey.
This project does not have any related story.
This project does not have any related emerging lesson.