A Practical Guide for Implementing the Digital Healthcare Equity Framework

Understand the Need and the Framework

What’s the Problem?

Despite the considerable opportunities that digital healthcare technologies provide to reduce healthcare inequities, substantial disparities due to race, ethnicity, and socioeconomic status still exist in access to and use of healthcare services.

Addressing this digital divide and its impact on the health of individuals and communities requires an approach that intentionally considers equity throughout the lifecycle of digital healthcare solutions.

About the Framework

The Evidence- and Consensus-Based Digital Healthcare Equity Framework guides users in intentionally considering equity in healthcare solutions that involve digital technologies. Download the framework to learn more about the evidence-and consensus-based approach to developing the framework, which included an environmental scan, engagement of the technical expert panel (TEP) in various forums, and reviews by internal and external advisors.

Download the Framework

The framework serves as a tool to help users and other stakeholders assess whether healthcare solutions that involve digital technologies are equitable at every phase of the digital healthcare lifecycle.

Key Principles

Six key principles guided development of the framework:

1. Ensure digital healthcare solutions that involve digital technologies ameliorate, not exacerbate, inequities.
2. Represent equity through person-centeredness.
3. Encourage inclusivity and participatory creation of digital healthcare solutions.
4. Support effective implementation in diverse settings.
5. Ensure specific attention to policy/regulatory relevance or impact of the proposed solutions.
6. Focus on impact and outcomes for patients, health systems, and communities.

Components of the Framework

The framework’s domains and subdomains were derived from synthesizing concepts identified from a scoping review, key informant interviews, and input from a 30-person technical expert panel.

Digital Healthcare Equity Domains and Subdomains

Key Aims

An intentional approach to considering equity throughout the digital healthcare lifecycle aims to promote improvement in

• Clinical outcomes such as mortality, morbidity, and health/quality of life
• Process outcomes such as care continuity, care coordination, and care quality
• Healthcare experiences such as patient satisfaction/engagement and provider satisfaction/engagement

It also seeks to achieve equitable access to and equity in the quality of healthcare solutions involving digital technologies.

General Recommendations

Any transformational change depends on strong leadership, organizational readiness, and ongoing systems support.

1. Assess Your Organizational Readiness

Perform an organization self-assessment of your readiness to start implementing the framework. The assessment should focus on 1) change management capabilities; 2) economic assessments, including costs and the potential return on investment related to equity intentionality; 3) health IT and data capabilities; and 4) leadership commitment to improving health equity.

2. Identify an Equity Champion

Identify an equity champion to help bring an equity lens to ongoing efforts to plan, develop, acquire, implement/maintain, and monitor/improve/perform equity assessment for healthcare solutions that involve digital technologies. The equity champion can help develop a business model and assess the economic sustainability and impact of the equity intentionality approach, which will help leadership systematically consider equity in long-term planning and resource allocation.

3. Develop a Diverse Workforce

Develop a diverse workforce capable of offering their lived experiences to ensure equity intentionality at an organizational level and for specific projects.

4. Build in Equity Assessments and Feedback Loops

Ensure that equity assessments and feedback loops are built in to processes related to healthcare solutions that involve digital technologies. The feedback loops should continuously assess and improve equity intentionality in healthcare solutions to achieve predefined success.

5. Track Whether Equity is Achieved

Track equity as part of your organizational metrics. For example, develop an equity dashboard to measure equity intentionality in key healthcare solutions that involve digital technologies during key phases of the digital healthcare lifecycle.

Where Are You in the Digital Healthcare Lifecycle?

Get a checklist with clear steps, ideas for engaging stakeholders, and real-world examples.

• Planning and Development
• Acquisition and Implementation/Maintenance
• Monitoring/Improvement/Equity Assessment

Equity Checklist for Planning and Development

Guidance for Digital Healthcare Developers and Vendors*

1. Identify and engage potential users of the digital healthcare solution, particularly those in historically marginalized demographic groups, to ensure it will meet the needs of the intended audience.

Each stage of the technology planning and development process represents an opportunity to intentionally address equity, including developing requirement specifications, design descriptions, and test plans. Developers and vendors are encouraged to employ principles of user-centered design, engaging potential users of the proposed solution.

Equity-related factors to consider in the planning and development phases include age, gender, race, ethnicity, primary language, sexual orientation, gender identity, socioeconomic characteristics, digital access, and digital literacy. For example, technology products should not assume cisgender identifiers and should allow for individuals to identify their gender, as appropriate.

One often-overlooked factor is functional disability status, which could be included as a voluntary self-identified demographic factor, including when/how the disability occurred. Just as other demographic factors impact beliefs and attitudes toward healthcare solutions that involve digital technologies, when someone acquired a disability, how long someone has experienced a disability, and the type of disability can affect those attitudes.

Real-World Example: Digital Determinants of Health 
Assess digital literacy among elderly patients using a survey, such as the eHealth Literacy Scale (eHEALS) tool, for a solution targeting specific patient populations. Having a clear understanding of an audience’s digital literacy levels will help inform the design of a solution.

For example, during the COVID-19 pandemic, there was a preponderance of Bluetooth-based digital healthcare solutions when the highest burden of the disease was on elderly populations who were less likely to be aware of or comfortable with this functionality. This resulted in a mismatch between the demographic and digital literacy characteristics of the users with the available technology.

Real-World Example: Sociodemographic Characteristics  
Assure the digital healthcare solutions accommodate the needs of patients with disabilities and those who are developing disabilities. Accordingly, consider proper font and color for the interface to accommodate visual impairment and employ voice-activated technology to accommodate hearing impairment.

2. Understand the cultural characteristics and beliefs of the communities for which a digital healthcare solution is proposed to identify potential barriers to using the proposed solution.

Identifying and collaborating with trusted partners in the community helps both to assess the digital capacity and infrastructure needs of the community and to understand the ways cultural beliefs in the community might impact the uptake of a solution.

Real World Example: Find opportunities to sit down with community members to understand their beliefs and lived experiences. For example, in designing a mobile application for African American men to connect with mental health services, you might hold roundtables with African American men to better understand how historical racism and mistreatment influence their trust of the U.S. healthcare system and their willingness to use a healthcare solution that requires the potential disclosure of sensitive information.

Real World Example: Build trust by investing in community partnerships that engage racial and ethnic groups with a disproportionate burden of health disparities. Investments for community engagement could include providing financial incentives for community-based organizations to collaborate with the developers during the planning and development of a digital healthcare solution. Engaging people from minority groups, such as African American adults and bilingual or Spanish-speaking Hispanic adults, in the designing process through community partnerships creates a “chain of trust” that helps potential users feel comfortable with the product.

3. Consider the impact of the proposed solution on digital equity in access, quality, and continuity of care in distinct healthcare settings.

Work with representatives of the health systems for which the solution is planned to be implemented, and map the clinical workflows. Digital healthcare solutions rarely follow analog workflows and can disrupt familiar clinical workflows. Thus, it is important to map workflows to identify where a digital solution is relevant and what strategies are needed to support change management for patients and providers.

Real World Example: Enable a new solution for remote patient monitoring to make use of cellular instead of Bluetooth connectivity so that the solution does not require Wi-Fi and assures wide access across different communities.

Real World Example: Design patient-facing digital technologies and software to be used by multiple people/ devices under one account to ensure wide access and to create interdependence between users and developers.

4. Assess whether the proposed solution serves as a facilitator (versus a barrier) to accessing and receiving high-quality care.

Hold roundtable discussions with different stakeholders and perform a root cause analysis of potential factors that could contribute to inequitable health outcomes that the proposed solution seeks to solve. Be proactive in thinking about what barriers the proposed solution might create for some subgroups of patients to access and receive high-quality care.

Real World Example: In developing a website that allows patients to sign up for vaccinations, hold discussions with patients, policymakers, and clinical providers to determine which patient subpopulations would experience improved access to high-quality care from the solution and which populations would experience additional barriers to quality healthcare.

Real World Example: To prevent implicit bias in the healthcare system affecting access to and use of a new solution, develop opt-out enrollment processes for eligible patients to remove dependence on clinician referral.

Real World Example: Data transparency can improve the quality of care. For a more transparent process, incorporate data analysis and interpretation tools in the patient-user interface of digital technologies to empower patients and caregivers to be more active in managing their personal health and healthcare.

Real World Example: To ensure widespread access, develop products and business models that target both safety net health systems and academic early adopters.31 An example of this is how OCHIN, a national network of community health organizations, has tailored electronic health record-based tools to help streamline digital and clinical workflows for community health centers.

Real World Example: To ensure risk-prediction models are not biased toward people with disabilities, be cautious about ableism inherent in many standardized measures of health. Embedded systemic ableism affects how people with disabilities receive medical equipment, such as receiving ventilators during the COVID-19 pandemic. During the pandemic, anticipated shortages in medical equipment such as ventilators led many States to draft or update their existing plans for crisis standards of care. The use of certain scoring systems that incorporate disability allocation during the eligibility determination process led to a decreased likelihood of individuals with disabilities receiving life-saving medical treatment during periods of limited availability.

Real World Example: Test medical devices on different subpopulations of patients to ensure that racial and ethnic variations do not result in the malfunction of medical devices and subsequently alter the medical care provided to those patients. For instance, pulse oximeters are racially biased and work less accurately on dark-skinned populations because melanin interferes with light-based pulse oxygen measurements. This bias may result in the provider missing hypoxemia events for dark-skinned patients.

5. Assess the technical characteristics of the proposed solution and whether those meet the current needs of potential users.

Implement proper methodologies, such as agile methodology, to ensure cycles of planning/ development are accompanied by seeking feedback from potential users of the proposed solution. Such approaches result in incremental deliverables and dynamic flexibility as the solution matures, helping ensure the solution’s technical characteristics consider the needs of the users.

Real World Example: The developer of patient-facing healthcare solutions may design multilingual interfaces to address the needs of diverse patient populations.

Real World Example: The developers may ensure that a new digital healthcare solution is compatible with other devices, offer translation extensions and plugins for digital health information and services, and offer multilingual and multimodal support that considers different levels of digital literacy.

6. Ensure that data are used equitably and transparently during the creation of a proposed solution and when a solution is capturing, generating, or transmitting data.

It is important to ensure datasets used to create, train, or test a proposed solution adequately represent the characteristics of the patient population (e.g., race, sexual orientation, gender) for which it was developed. Being transparent with users that are implementing or using a solution about what and how data were used to develop, train, or test a solution is equally important.

Develop policies, procedures, and controls that take patient privacy into account by ensuring data captured, generated, or transmitted by a solution are accurate, transparent, secure, and interoperable. Regular review of privacy protocols and content in close collaboration with are presentative sample of users will help developers to iterate the policies and protocols as needed to maximize privacy and control by all users.

Real World Example: To build interest and trust among patients, a technology vendor planning and developing a remote patient monitoring solution may allow patients to actively approve all data transmitted to clinicians.

Real World Example: Technology vendors may engage in Health Level Seven (HL7 community forums; a set of international standards for the transfer of clinical and administrative data between software applications used by various healthcare providers) to: 1) lobby for the inclusion of data on social needs and social determinants of health in Fast Healthcare Interoperability Resources standards (FHIR; a standard used to access and exchange healthcare data) and 2) participate and keep up to date with those standards.

Engaging Other Stakeholders

Patients/Caregivers, Patient Advocates, and Community Champions

• Collaborate with relevant patient and community organizations to learn about their needs, goals, or potential challenges regarding the proposed digital healthcare solution.

Health Systems, Clinical Providers, Purchasers, Health Plans, and Public Health

• Seek information on their digital and clinical workflows in a variety of different settings to ensure the successful development and implementation of the digital healthcare solution.
• Seek information related to the reimbursement of any new services provided by the proposed solutions.

Policymakers

Set up regular roundtable discussions with policymakers to share challenges related to the structural barriers in providing healthcare solutions involving digital technologies, and suggest potential policies that could address those barriers.

Equity Checklist for Acquisition and Implementation/ Maintenance

Guidance for Health Systems, Health Plans, and Clinical Providers

1. Adopt a digital inclusion-informed strategy regarding the acquisition, implementation, and maintenance of healthcare solutions that involve digital technologies to reduce and eliminate barriers.

Adopting a digital inclusion-informed strategy helps the health systems, health plans, and clinical providers recognize their community’s sociodemographic, social, and digital determinants of health. It also helps provide proper support to their patients in their initial implementation and sustained maintenance or use of healthcare solutions that involve digital technologies, as well as their monitoring, improvement, and equity assessment. Such intentional strategy and investment in digital equity will help to reduce and eliminate historical, institutional, and structural barriers to access and use healthcare solutions that involve digital technologies. Digital inclusion refers to “the activities necessary to ensure that all individuals and communities, including the most disadvantaged, have access to and use of Information and Communication Technologies

Real World Example: Focus on patient training as it relates to security and privacy during the implementation/ maintenance phase to account for varied digital literacy levels.

Real World Example: Assess and be mindful of available and affordable broadband internet services and internet-enabled devices in the communities in which the digital healthcare solution is provided.

Real World Example: Make referrals to available programs such as the Affordable Connectivity Program, an FCC benefit program that helps ensure that households can afford the broadband they need for work, school, and healthcare.

2. Consider a participatory and multisectoral collaboration for proper acquisition, implementation, and maintenance of healthcare solutions that involve digital technologies.

Identify hard-to-reach patients and develop strategies for multisectoral outreach initiatives.

Real World Example: During the COVID-19 pandemic, the Pima County Health Department in Tucson, AZ, initiated a campaign on COVID-19 vaccination and other mitigation strategies. They leveraged local data to identify racial and ethnic minority populations at the highest risk for health disparities and low health literacy and populations not currently reached through existing public health campaigns. They identified challenges related to the delivery of information to those with housing issues (e.g., homelessness and housing insecurity and lack of a permanent address). They worked closely with the Department of Housing and Community Development for this outreach.

3. Consider the impact of the implemented solution on digital equity (access to and quality of care, and care continuity) across different types of health systems.

Perform a systematic assessment regarding the impact of healthcare solutions to identify potential factors impacting access to and quality of care affected by the solutions. Additionally, design initiatives to assure equitable access to high-quality and continued care.

Real World Example: To assure wide access to a new healthcare solution that involves digital technologies, train clinical providers to offer access to all patients, rather than access based on presumed use of the technology, and to encourage patients to use the solution as part of standard care.

Real World Example: Provide relevant digital health literacy sensitivity training to clinical providers and health professionals to assist patients in navigating the healthcare solutions that involve digital technologies, and offer tailored solutions specific to their patient’s level of digital health literacy.

Real World Example: Implement digital platforms in different languages as required, and provide digital navigators who will assist the patients in gaining access to and using healthcare solutions that involve digital technologies to assure equitable health quality and continued care.

Real World Example: Invest in community-based organizations or local partnerships (e.g., libraries) to make devices with the capability for internet or cellular data access freely available in underserved communities (e.g., computers, tablets, smartphones).

4. Adopt strategies that guarantee a new healthcare solution involving digital technologies serves as a facilitator and not as a barrier to accessing and receiving high-quality care.

Partner with community organizations to identify barriers and facilitators to accessing and receiving high-quality care by using healthcare solutions that involve digital technologies. This information would help to adopt strategies that address the barriers and reemphasize the facilitators to equitable and high-quality care.

Real World Example: Provide continuous quality technical support for patients with different technical skills and levels of eHealth literacy.

5. Assess the technical characteristics of the solution and how those match the current needs of potential users.

Implement different approaches for seeking feedback from potential users of the solution. Such approaches may result in the incremental implementation of new solutions in existing clinical and digital workflows, helping to ensure the technical characteristics of the solution consider the needs of the users.

Real World Example: Invest considerable resources into acquiring and integrating technology platforms that connect providers and patients with community-based organizations into the electronic health records, train clinical and administrative staff on new platforms, and ensure that platforms become a routine component of their clinical workflow.

Real World Example: Invest in and implement patient-facing solutions (e.g., patient portals and mobile health applications) that address the needs of patients from medically underserved communities.

Real World Example: Develop workflows that allow clinical teams to engage with diverse patients across healthcare solutions that involve digital technologies, such as telehealth.

6. Before acquiring a healthcare solution, consider how it was developed, where and how it will be implemented and maintained, and how it will use, produce, or transmit data.

Some solutions, such as algorithms, can be developed with biased data or modeled on incorrect assumptions. As a result, outputs from these solutions can perpetuate biases and inequities in care. Therefore, before acquiring a new healthcare solution, consider how it was developed and where and how it will be deployed, monitored, and maintained to address or eliminate the potential for new and emerging biases. Specifically, consider if data were used equitably and transparently in the development of a solution. Prior to implementing a solution, consider the accuracy, security, and interoperability of the data it will capture, generate, and transmit. Review and communicate processes for how a solution will be monitored and maintained once deployed.

Regular review of privacy protocols and content in close collaboration with a representative sample of users is vital. Plan to update policies and protocols as needed to maximize patient privacy and control by all users.

Real World Example: Wang et al. developed a bias evaluation checklist that allows model developers and health care providers a means to systematically appraise a model’s potential to introduce bias prior to it being developed or deployed.

Real World Example: Pierson et al. implemented a machine learning-based algorithm to measure the severity of osteoarthritis by using knee x-rays to predict pain experienced by patients. This approach dramatically reduced unexplained racial disparities in patient pain, relative to standard measures of severity graded by radiologists. The algorithmic predictions better captured underserved patients’ pain.

Engaging Other Stakeholders

Patients/Caregivers, Patient Advocates, and Community Champions

• Collaborate with patient advocacy organizations and learn about their needs, goals, and challenges. Prioritize organizations that represent populations that have historically carried a disproportionate burden of health disparities.

Health Systems, Clinical Providers, Purchasers, Health Plans, and Public Health

• Provide information to developers and vendors on the healthcare system’s digital and clinical workflow at a variety of settings to ensure the successful development and implementation of the digital health solutions.
• Provide information related to payer reimbursement of the new services provided by the proposed solutions to help developers and vendors plan for solutions that provide more equitable and high-quality care.

Policymakers

• Lobby for future regulations and programs that would include metrics that assess the use of digital healthcare solutions by medically underserved communities.
• Endorse Federal policies that guide culturally appropriate care.
• Encourage policies that expand broadband access.

Equity Checklist for Monitoring/Improvement/Equity Assessment

Guidance for Health Systems, Health Plans, and Clinical Providers

1. Identify the characteristics of the populations that are using a healthcare solution that involves digital technologies, and identify populations presently excluded, not benefiting, or not participating at the desired or same rates as others.

Assessing the characteristics of the populations that are affected by the solution helps to identify comparison populations or reference points for monitoring, improvement, and equity assessment. It also helps to identify intersectionality or overlaps of characteristics in ways that expose those populations to relatively greater inequities.

Real World Example: In 2022, the California Department of Managed Health Care adopted standard health equity and quality measures for health plans to ensure the equitable delivery of high- quality healthcare service for all enrollees. The health plans are now required to stratify reported data by race and ethnicity to demonstrate delivery of equitable quality of healthcare.

2. Use a participatory approach to collect input from affected community members about the healthcare solutions that involve digital technologies.

Identify experts, including former or current solution participants/beneficiaries, members of communities affected by the solution (e.g., patients using a patient portal platform), staff members who work with participants/beneficiaries, or affected communities, subject matter experts such as researchers, or staff in different organizations. Develop a comprehensive process including different sources of expert input from listening sessions to surveys, interviews, and focus groups by experts in the field or advocacy groups. In this participatory approach, ensure inclusivity for different experts, especially individuals or communities that have historically been excluded or disempowered in decision making.

Real World Example: Provide different translational services or accommodations for people with disabilities to ensure inclusivity in the process of collecting input from different experts.

Real World Example: Consider different methods of seeking input to accommodate people with different communication preferences or time constraints, or transportation constraints (e.g., a survey or a focus group in person or via Zoom).

Real World Example: Identify strategies to decrease power dynamics and to ensure that experts are comfortable providing candid input to perform a comprehensive monitoring, improvement, and equity assessment.

Real World Example: Provide transparency about how input will be shared and used.

Real World Example: Use appropriate data-collection methods that center community perspectives and expertise, and report back to community members on how data are used. Community members may want to participate in gathering data about their own community and could provide insights on the best ways to do so.

3. Consider the impact of the healthcare solution on digital equity (access to and quality of care, and care continuity) across different types of health systems.

Continuously assess how the solution impacts access to and quality of care, especially among individuals or communities who have historically been excluded. Design initiatives to assure equitable access to high-quality and continued care.

Real World Example: Establish long-term partnerships with community organizations with expertise in training community members in digital literacy skills and facilitating connectivity for continued training of community members interested in using the new technology.

Real World Example: Advocate for Medicare/Medicaid reimbursement for community health workers to support remote patient monitoring workflows. This effort would ensure the continued use of such solutions for patients with challenges related to access to care.

4. Adopt strategies that guarantee a new healthcare solution that involves digital technologies serves as a facilitator, not a barrier, to accessing and receiving high-quality care.

Partner with experts, including former or current solution participants/beneficiaries and members of communities affected by the solution, and inquire about their views on the benefits and burdens involved in the continued use of the proposed solution. Further, ask for experts’ perceptions of barriers to continual use of the solutions and their views on current or potential burdens or barriers that are more severe for certain population groups.

Real World Example: Partner with experts to identify potential community members who are excluded, are not participating, or are not benefiting from the new solution at desired rates or at the same rates as others.

Real World Example: Acquire and implement applications and online content to enable and encourage self- sufficiency, participation, and collaboration among those users.

5. Identify information sources and gaps in available data for a comprehensive monitoring, improvement, and equity assessment of digital technologies.

Identify quantitative and qualitative data sources for these processes. Quantitative data, such as survey data, shed light on the magnitude and prevalence of problems related to access and use of solutions, as well as opportunities for improvement. Qualitative data sources, such as an interview or focus group data, increase understanding of context and help to interpret and understand quantitative data.

Real World Example: Assess whether available data is disaggregated by relevant variables such as race, ethnicity, income, and geographic areas. Such data helps the monitoring and improvement process to include an estimate of the equity impacts of a new healthcare solution that involves digital technologies.

Real World Example: Collaborate with a technology developer and vendor of a new solution to develop functionalities for disaggregated data collection among patients who are medically underserved for a comprehensive assessment of the equity impacts of a solution.

Real World Example: Carefully assess patterns of missing data from communities. Some patterns of missing data may be due to the community fatigue from continuous assessments and data collections.

Real World Example: Consider using “Data Walks” as a means of sharing key data and research findings with stakeholders in small groups who interpret the data, and then collaborate to improve the solutions. Using data sharing as a platform for collaboration, a Data Walk can help to ensure a more robust analysis and understanding of the data, inform better policies that address both the strengths and needs of a particular community or population, and inspire individual and collective action among community members.

Engaging Other Stakeholders

Patients/Caregivers, Patient Advocates, and Community Champions

• Collaborate with patient advocacy organizations and learn about their needs, goals, and challenges. Prioritize organizations that represent populations that have historically carried a disproportionate burden of health disparities.

Health Systems, Clinical Providers, Purchasers, Health Plans, and Public Health

• Provide information to developers and vendors on the healthcare system’s digital and clinical workflow at a variety of different settings to ensure the successful development and implementation of the digital health solutions.
• Provide information related to payer reimbursement of the new services provided by the proposed solutions to help developers and vendors plan for solutions that provide more equitable and high-quality care.

Policymakers

• Lobby for future regulations and programs that would include metrics that assess the use of digital healthcare solutions by medically underserved communities.
• Endorse Federal policies that guide culturally appropriate care.
• Encourage policies that expand broadband access.