Automating Assessment of Obesity Care Quality (California)

Project Final Report (PDF, 223.85 KB) Disclaimer

Project Details - Ended

Project Categories

Summary:

The prevalence of obesity has doubled over the past two decades, with two-thirds of the adult population now overweight and one-third obese. Routine ambulatory care encounters present an opportunity to introduce lifestyle changes with patients and discuss options of medical and surgical treatments. However, despite the availability of published guidelines on preventing, diagnosing, and treating obesity, most clinicians have been slow to respond to this public health problem.

This project developed a set of nine quality measures to evaluate obesity care based on the National Heart, Lung, and Blood Institute (NHLBI) guidelines. Natural language processing was used to automatically extract electronic medical record (EMR) data for the measures. This methodology was implemented to comprehensively assess the quality of adult obesity care in primary care settings.

The specific aims of this project were as follows:

  • Develop obesity care quality measures based on updated NHLBI guidelines to evaluate obesity care performance in primary care. 
  • Use comprehensive EMR data to develop and validate an automated (generalizable and scalable) method for applying the measures identified in the first aim. 
  • Apply the method developed in the second aim to assess ambulatory obesity care quality in two distinct health plans representing diverse patient populations and care practices. 
  • Evaluate the association between measures of obesity guideline adherence to recommended obesity care processes and clinical outcomes and provider characteristics. 

A retrospective analysis was done of EMR data from a mid-sized health maintenance organization and a consortium of safety-net clinics located in the Pacific Northwest using the new tool. The measures were compared against chart review of more than 900 patients to assess accuracy. Outcomes associated with the delivery of the recommended care were assessed.

EMR-based advice and diet and exercise plan measures performed best relative to the reference standard. Waist circumference and food and exercise diary education measures occurred too infrequently and so were not able to be evaluated well in this project. The project team determined that documentation of body mass index, readiness, and followup visit measures will need further work to resolve discrepancies between the EMR and chart review measures. A small but significant association was noted between guideline-recommended care and weight loss in patients who were overweight or obese.

Future research in this area would involve determining which, if any, EMR-based measures need refining of specifications to identify measures of interest more accurately, and which perform better in terms of locating information that is less accessible to manual chart reviewers.

Automating Assessment of Obesity Care Quality - 2012

Summary Highlights

  • Principal Investigator: 
  • Funding Mechanism: 
    PAR: HS08-270: Utilizing Health IT to Improve Health Care Quality Grant (R18)
  • Grant Number: 
    R18 HS 018157
  • Project Period: 
    September 2009 – March 2013
  • AHRQ Funding Amount: 
    $1,194,761
  • PDF Version: 
    (PDF, 283.13 KB)

Summary: Obesity and its public health effects are an increasing burden on the health care system. This project proposed to develop, implement, and evaluate a routine automated method to assess outpatient obesity care quality using measures from comprehensive electronic medical record (EMR) data based upon the National Heart, Lung, and Blood Institute (NHLBI) obesity care guidelines.

The study team is investigating associations between obesity care delivery steps and clinical outcomes known or suspected to be accelerated by obesity. Measures to evaluate the association include reasons for visit; orders; referrals; diagnosis codes; vital signs; text clinical notes pertaining to weight (e.g. body mass index) and weight-loss counseling; and other obesity intervention efforts. The project team is using percent-change in body weight as the primary outcome measure.

Retrospective EMR data from Kaiser Permanente Northwest (KPNW), a midsized health maintenance organization, and Oregon Community Health Information Network (OCHIN), a consortium of federally qualified health centers, is being used to evaluate the association between obesity guideline adherence and clinical outcomes. Information from both structured and free-text fields will be used. Free-text fields are being automatically coded using natural-language processing computer software. Data produced under the automated method of quality measurement is being compared to medical record reviews performed by abstractors in order to assess the validity of the automated system.

The automated system is being applied to patient populations of KPNW and OCHIN, which total more than 350,000 adults to determine: 1) the proportion of overweight or obese patients who are receiving advice, counseling, weight-loss program referral, prescription medication, and other care recommended by the guidelines; 2) correlates of overweight and obesity diagnosis and treatment guideline adherence including patient characteristics, comorbidity status, provider characteristics, and health system characteristics; and 3) changes in health status as a function of guideline adherence for obese patients.

Specific Aims:

  • Develop obesity care quality measures based on updated NHLBI guidelines to evaluate obesity care performance in primary care. (Achieved)
  • Use comprehensive EMR data to develop and validate an automated (generalizable and scalable) method for applying the measures identified in the first aim. (Ongoing)
  • Apply the method developed in the second aim to assess ambulatory obesity care quality in two distinct health plans representing diverse patient populations and care practices. (Ongoing)
  • Evaluate the association between measures of obesity guideline adherence to recommended obesity care processes and clinical outcomes and provider characteristics. (Ongoing)

2012 Activities: During 2012, the study team staged all data necessary for applying the measurement process. They established eligibility criteria that: 1) requires patients’ continuous enrollment; and 2) excludes patients with pregnancies, any indication of non-benign cancer history, palliative or hospice care, or eating disorders. Patients from these sub-populations were excluded because they have unique body weight issues. The team has drawn the primary care encounter data for the resultant population
for the observation period beginning January 1, 2007 at KPNW, yielding approximately 125,000 eligible patients; and at OCHIN, yielding approximately 30,000 eligible patients. The study staff completed the KPNW and OCHIN chart reviews and the validation study. Final activities focused on the data analysis and manuscript writing.

As last self-reported in the AHRQ Research Reporting System, project progress and activities are on track in some respects but not others and project budget funds are somewhat underspent, due to decreased staffing on the project. Dr. Hazlehurst is using a 1 year no-cost extension to complete data collection and analysis and manuscript writing.

Preliminary Impact and Findings: This project has no findings to date.

Target Population: Adults, Chronic Care*, Obesity

Strategic Goal: Develop and disseminate health IT evidence and evidence-based tools to improve health care decisionmaking through the use of integrated data and knowledge management.

Business Goal: Knowledge Creation

* This target population is one of AHRQ’s priority populations.

Automating Assessment of Obesity Care Quality - 2011

Summary Highlights

  • Principal Investigator: 
  • Funding Mechanism: 
    PAR: HS08-270: Utilizing Health IT to Improve Health Care Quality Grant (R18)
  • Grant Number: 
    R18 HS 018157
  • Project Period: 
    December 2009 - March 2013
  • AHRQ Funding Amount: 
    $1,194,761
  • PDF Version: 
    (PDF, 198.74 KB)

Summary: Obesity and its public health effects are an increasing burden on the health care system. This project proposes to develop, implement, and evaluate a routine, automated method to assess outpatient obesity care quality using measures from comprehensive electronic medical record (EMR) data based upon the National Heart, Lung, and Blood Institute (NHLBI) obesity care guidelines.

The study team is investigating associations between obesity care delivery steps and clinical outcomes known or suspected to be accelerated by obesity. Measures to evaluate the association include: reasons for visit; orders; referrals; diagnosis codes; vital signs; text clinical notes pertaining to weight (BMI) and weight loss counseling; and other obesity intervention efforts. The project team is using percent change in body weight as the primary outcome measure.

Retrospective EMR data from Kaiser Permanente Northwest (KPNW), a midsized health maintenance organization and Oregon Community Health Information Network (OCHIN), a consortium of federallyqualified health centers, is being used to evaluate the association between obesity guideline adherence and clinical outcomes. The project is using Kaiser Permanente's Epic-based EMR, HealthConnect, and OCHIN's EMR, EpicCare. Information from both structured and free-text fields will be used. Freetext fields are being automatically coded using natural-language processing computer software. Data produced under the automated method of quality measurement is being compared to medical record reviews performed by abstractors in order to assess the validity of the automated system.

The automated system is being applied to patient populations of KPNW and OCHIN, which total more than 350,000 adults to determine: 1) the proportion of overweight or obese patients who are receiving advice, counseling, weight-loss program referral, medication prescription, and other care recommended by the guidelines; 2) correlates of overweight and obesity diagnosis and treatment guideline adherence including patient characteristics, comorbidity status, provider characteristics, and health system characteristics; and 3) changes in health status as a function of guideline adherence for obese patients.

Specific Aims:

  • Develop obesity care quality measures based on updated NHLBI guidelines to evaluate obesity care performance in primary care. (Achieved)
  • Use comprehensive EMR data to develop and validate an automated (generalizable and scalable) method for applying the measures identified in the first aim. (Ongoing)
  • Apply the method developed in the second aim to assess ambulatory obesity care quality in two distinct health plans representing diverse patient populations and care practices. (Ongoing)
  • Evaluate the association between measures of obesity guideline adherence to recommended obesity care processes and clinical outcomes and provider characteristics. (Ongoing)

2011 Activities: Study staff identified the content-specific rules and codes required to distinguish the relevant clinical events (e.g., the order codes used at each site that indicate "obesity counseling"), and codified the text statements clinicians use to satisfy the measures (e.g., "advised pt to lose wt"). These statements are being built into the knowledge base of the automated program that identifies the relevant clinical events of the quality measure set.

An abstraction form and process were developed to enable chart review to validate the measures. A random sample of approximately 450 study patients, stratified by age, sex, and BMI, was selected at each site. Both KPNW and OCHIN chart reviews were completed with a 10 percent sample validation, using a practicing primary care clinician to ensure the quality of chart reviews.

Early in 2011, the project slowed significantly because the OCHIN study site was reworking their datasharing arrangements with their participating clinics. This put on hold all access to the data by the non- OCHIN staff. The data agreements were subsequently executed and the project team was able to view patient data and begin the key tasks of chart review and quality measure implementation.

All data necessary for applying the measurement process has been staged. Primary care encounter data was drawn for the resultant population for the observation period beginning in January 2007 in KPNW (yielding roughly 125,000 eligible patients), and OCHIN (yielding roughly 25,000 eligible patients).

Initially, the team planned a 24-month evaluation period because they interpreted the NHLBI guideline to mean that BMI and waist circumference were to be documented every 2 years. They have re-interpreted the guideline to mean that BMI should be assessed at all primary care visits for all patients, and waist circumference at all primary care visits for obese and overweight patients. This change in the method reduced the need for evaluation from a 24- to a 1-month window.

As last self-reported in the AHRQ Research Reporting System, project progress and activities are on track in some respects but not others and project funds are significantly underspent due to the previously described delay of data-sharing agreements with participating clinics. The project is using a 1-year no-cost extension to complete the project.

Preliminary Impact and Findings: This project has no findings to date.

Target Population: Adults, Chronic Care*, Obesity

Strategic Goal: Develop and disseminate health IT evidence and evidence-based tools to improve health care decisionmaking through the use of integrated data and knowledge management.

Business Goal: Knowledge Creation

* This target population is one of AHRQ's priority populations.

Automating Assessment of Obesity Care Quality - 2010

Summary Highlights

  • Principal Investigator: 
  • Funding Mechanism: 
    PAR: HS08-270: Utilizing Health IT to Improve Health Care Quality Grant (R18)
  • Grant Number: 
    R18 HS 018157
  • Project Period: 
    December 2009 – May 2011
  • AHRQ Funding Amount: 
    $1,194,761
  • PDF Version: 
    (PDF, 583.91 KB)


Target Population: Adults, Chronic Care*, Obesity

Summary: Obesity and its public health effects are an increasing burden on the health care system. This project proposes to develop, implement, and evaluate a routine, automated method to assess outpatient obesity care quality using measures from comprehensive electronic medical record (EMR) data based upon the National Heart, Lung, and Blood Institute (NHLBI) obesity care guidelines.

The study team will use reasons for visit, orders, referrals, diagnosis codes, laboratory test values, and text clinical notes pertaining to weight loss counseling and other obesity intervention efforts to investigate associations between obesity care delivery steps and clinical outcomes known or suspected to be accelerated by obesity. Percent change in body weight was selected as the primary outcome measure.

Retrospective EMR data from both a midsized health maintenance organization and a consortium of Federally Qualified Health Centers will be used to evaluate the association between obesity guideline adherence and clinical outcomes. The project will use Kaiser Permanente’s Certification Commission for Health Information Technology-certified Epic-based EMR HealthConnect and Oregon Community Health Information Network’s Epic-based EMR EpicCare. Information from both structured and free-text fields will be used. Free-text fields will be automatically coded using natural language processing computer software. Data produced under the automated method of quality measurement will be compared to medical record reviews performed by abstractors in order to assess the validity of the automated system.

The automated system will be applied to two diverse patient populations totaling more than 350,000 adults to determine: 1) the proportion of overweight or obese patients who are receiving advice, counseling, weight loss program referral, medication prescription, and other care recommended by the guidelines; 2) correlates of overweight and obesity diagnosis and treatment guideline adherence including patient characteristics, comorbidity status, provider characteristics, and health system characteristics; and 3) changes in health status as a function of guideline adherence for obese patients.

Specific Aims:
  • Develop obesity care quality measures based on updated NHLBI guidelines to evaluate obesity care performance in primary care. (Achieved)
  • Use comprehensive EMR data to develop and validate an automated (generalizable and scalable) method for applying the measures identified in the first aim. (Ongoing)
  • Apply the method developed in second aim to assess ambulatory obesity care quality in two distinct health plans representing diverse patient populations and care practices. (Ongoing)
  • Evaluate the association between measures of obesity guideline adherence to recommended obesity care processes and clinical outcomes and provider characteristics. (Upcoming)

2010 Activities: The study team developed a working draft of the Obesity Care Quality (OCQ) Measure Set that defines the following:

  • Study population, including reasons for exclusion.
  • Nine distinct measures of obesity care quality, including criteria for qualification and measurement.
  • Risk factors that will be assessed to enable measurement of care for those at heightened risk for adverse health consequences of obesity.

Although the working draft of the OCQ Measure Set will be revised as needed, the associated aim was considered achieved. The team updated their data extraction process and, with a working draft of the OCQ Measure Set developed, have begun to identify the content-specific rules and codes required to identify the relevant clinical events, such as the order codes used at each site, that indicate "obesity counseling."

Progress was made in identifying the data necessary to assess ambulatory obesity care quality in the two health systems. Summary analyses to understand numbers of patients meeting basic eligibility criteria were created and the project has refined and generalized the measurement methodology utilized in a previous Agency for Healthcare Research and Quality-funded study.

Grantee's Most Recent Self-Reported Quarterly Status (as of December 2010): Project progress is completely on track with all milestones being met and progressing on schedule. Project spending is somewhat underspent, due to a slower than expected startup. It is anticipated to be on track in the upcoming year.

Preliminary Impact and Findings: The project does not have any findings to date.

Strategic Goal: Develop and disseminate health IT evidence and evidence-based tools to improve health care decisionmaking through the use of integrated data and knowledge management.

Business Goal: Knowledge Creation

*AHRQ Priority Population.

Automating Assessment of Obesity Care Quality - Final Report

Citation:
Hazlehurst B. Automating Assessment of Obesity Care Quality - Final Report. (Prepared by Kaiser Foundation Research Institute under Grant No. R18 HS018157). Rockville, MD: Agency for Healthcare Research and Quality, 2013. (PDF, 223.85 KB)

The findings and conclusions in this document are those of the author(s), who are responsible for its content, and do not necessarily represent the views of AHRQ. No statement in this report should be construed as an official position of AHRQ or of the U.S. Department of Health and Human Services.
Principal Investigator: 
Document Type: 
Population: 
Medical Condition: 
This project does not have any related resource.
This project does not have any related survey.
This project does not have any related project spotlight.
This project does not have any related survey.
This project does not have any related story.
This project does not have any related emerging lesson.