Taking the Garbage Out: Addressing AI Bias in Quality Measurement

J Clin Pathways. 2024;10(6):26-29. doi:10.25270/jcp.2024.10.01

Recent literature has shed light on the expanding role of artificial intelligence (AI) in health care, particularly for its use in quality measurement. Studies demonstrate AI’s potential to significantly enhance health care delivery by improving clinical op­erations, bolstering quality and safety, and refining the mechanisms of quality re­porting.^1,2 Projections suggest that AI adoption could save the health care industry between $200 and $360 billion annually,³ thereby underscoring its potential to yield substantial economic and operational benefits.

However, alongside these optimistic projections, a critical discourse has emerged regarding the potential pitfalls of AI integration in health care, particu­larly concerning bias. For example, research has documented instances in which AI applications in health care exacerbate, rather than mitigate, disparities,⁴ indi­cating a need for nuanced exploration of AI’s limitations, especially biases that may skew AI-derived insights.

Recent policy initiatives aim to address the challenges of AI in health care. The Biden-Harris Administration’s Executive Order on the Safe, Secure, and Trustwor­thy Development and Use of Artificial Intelligence, for example, promotes a frame­work to help ensure AI is ethically applied in public policy, including in health care. This Executive Order highlights the importance of developing AI tools that are not only effective, but also equitable and transparent in their function.⁵ In addition, the US Centers for Medicare & Medicaid Services (CMS) recently convened an informa­tion session titled “AI in Quality Measurement.”⁶ The CMS session contributed to our ongoing understanding of AI in quality measurement by underscoring the in­creasing significance of leveraging advanced computational technologies to enhance the accuracy and efficiency of measurement, reflecting a broader push within the health care community toward innovation-driven solutions.

The timing is right for stakeholders to critically assess AI’s potential to trans­form health care quality measurement. This assessment must rigorously address the dual objectives of harnessing AI’s transformative potential for quality measure­ment while vigilantly curbing its capacity to reinforce existing health care biases. To that end, this article focuses on exploring the use of AI in health care quality measurement and identifying tactics to mitigate bias in AI-driven quality assessment.

AI integration in health care represents a transformative shift, including in quality measurement, which focuses on evaluating the effectiveness, efficiency, and equity of health care services. Quality measures help quantify health care processes, outcomes, patient perceptions, organizational structures, and systems that are associated with delivering high-quality health care.7 AI tools in quality measurement encompass a range of technologies and applications designed to enhance precision, efficiency, and effec­tiveness of quality measurement processes. Examples of these tools include AI-enabled data collection systems that automate and refine the gathering of clinical information,^8,9 AI measure­ment and analytical tools that assess and interpret quality data,⁸ and AI-driven analytics platforms that can integrate and ana­lyze large datasets to identify trends and actionable insights.¹⁰ AI in the context of quality measurement is tasked with not only gathering and analyzing data, but also ensuring that the insights derived are devoid of bias that could skew outcomes. Such an endeavor necessitates a deep understanding of both the potential and the limitations of AI technologies in accurately capturing and interpreting complex health care data.

AI, particularly large language models (LLMs), is increas­ingly pivotal in enhancing the accuracy, reliability, and effi­ciency of health care quality measurement.¹⁰ LLMs are trained on large datasets, enabling them to comprehend and generate natural language, along with other forms of content.¹¹ By le­veraging LLM capabilities, AI can automate the extraction of complex clinical concepts from patient visit notes, ensuring faster coding with greater precision, while significantly reduc­ing administrative burden.^8,9 This advancement enables clini­cians, hospitals, and health plans to more efficiently report data within value-based programs, optimizing both patient care and resource use.

Furthermore, AI has the potential to transform quality measurement by providing novel insights and revealing con­nections between different metrics, potentially creating new avenues for improving clinical outcomes and facilitating a more equitable health care system. For instance, AI can leverage pat­terns and correlations that may not be immediately obvious to human analysts,^2,10,12 which may highlight areas where quality measures might unjustifiably penalize or favor certain groups. Additionally, AI can detect potential harms and unintended consequences of existing quality measures, offering insights to refine and enhance measures.¹³ This ensures that measures are valid, reliable, and can be applied fairly for a diverse popula­tion. Consequently, AI not only supports existing quality mea­surement frameworks, but also sets the stage for developing in­creasingly sophisticated metrics that more accurately reflect the nuances of patient care and health system performance.

Although AI offers significant benefits by enhancing data fi­delity and enabling real-time analytics, it also poses challenges, such as data privacy concerns,⁹ the risk of algorithmic bias,¹⁴ and the need for substantial training on datasets that accurately represent diverse populations.¹⁵ Literature highlights that with­out appropriate training on representative datasets, AI systems might replicate existing disparities in health care provision or introduce new biases, thereby compromising the integrity of quality assessments.^15-17 Most AI tools used for quality mea­surement are trained on clinical data that reflect the patterns of the majority population, which may overlook the unique needs and perspectives of minority groups.⁸ Additionally, the financial burden associated with accessing these large datasets and preparing them for use in AI tools is substantial. Organi­zations must invest in costly data cleaning and preprocessing to ensure datasets are suitable for AI models.^18,19 Strategies to reduce these costs, such as collaborative data sharing initiatives and funding support for data infrastructure development, are important considerations.

The complex, opaque nature of AI algorithms can impede their acceptance, necessitating clear communication and rigor­ous validation protocols to build confidence in their use. There­fore, a focused analysis of AI’s application in quality measure­ment is imperative to navigate these challenges effectively and to harness AI’s full potential in improving health care quality.

Successful deployment of AI to support quality measurement requires a comprehensive, multi-stakeholder strategy dedicated to understanding, identifying, and mitigating bias. Below are tactics to address bias in AI-driven quality measurement.

Mitigating Bias in AI Systems Through Comprehensive Training Guidelines

To fully harness the transformative potential of AI in health care quality measurement, it is crucial to implement com­prehensive guidelines for training AI systems that rigorously address potential biases. The process of developing these guide­lines should be deliberate and emphasize the collection and utilization of diverse datasets that represent the broad spectrum of real-world patient demographics, medical conditions, and socioeconomic backgrounds. The guidelines should be peer reviewed, published, and continuously updated to reflect ad­vancements in AI technology and emerging insights from on­going research. Such data diversity is essential for mitigating biases and enhancing the effectiveness of AI in stratifying and evaluating quality measures across various patient groups.

The guidelines should be crafted through engagement with a multi-stakeholder group that includes AI developers, health care providers, medical societies, health plan representatives, patients and patient advocates, quality measurement experts, ethicists, and policymakers. Together, these stakeholders can ensure that AI models are not only technically proficient, but also ethically sound and socially responsible. For instance, AI and quality measure developers must work closely with health care professionals to understand the clinical implications of bi­ases and to integrate clinical insights into AI training processes. Meanwhile, ethicists and patient advocates can ensure that AI systems uphold patient dignity and do not inadvertently per­petuate health care disparities.

Moreover, policies mandating adherence to comprehensive training guidelines for AI models are necessary. These policies could require regular bias auditing and re-training of AI mod­els to ensure that AI systems evolve in response to new data and insights. Additionally, policies promoting transparency in AI operations, including required disclosure of datasets and algorithms used as well as actions taken to mitigate bias, are critical. There is a growing recognition of the need for policy frameworks to manage AI in health care. For example, over 50 lawmakers in the House and Senate have recently advocated for CMS to increase oversight of AI use in Medicare Advantage coverage decisions.20 By integrating robust training protocols through policy efforts, AI systems can provide more accurate, fair, and inclusive assessments of health care quality, thereby supporting enhanced patient outcomes and contributing to the integrity and reliability of quality measurement.

Educating Stakeholders on AI’s Role in Quality Measurement

Key stakeholders, including policymakers, measure developers, and health care providers, must be well informed of the nu­ances of AI applications as its use expands within health care quality measurement. Continuous education and training are vital to help these stakeholders understand the potential biases in AI tools, methodologies used in developing AI models, and critical strategies needed to effectively use AI-generated data.

For policymakers, specialized workshops and briefing ses­sions could provide guidance in their efforts to govern the use of AI in health care, ensuring these technologies serve the pub­lic interest without reinforcing existing disparities. These edu­cational efforts should focus on how AI technologies and tools should be designed and monitored to prevent bias and ensure equitable health outcomes. Similarly, quality measure develop­ers and health care providers require ongoing training to stay current with the latest advancements and ethical considerations in AI. Training may include professional development courses, certifications, and seminars that delve into the application of AI in clinical settings, emphasizing ethical considerations, bias mit­igation, and nuanced interpretation of AI-generated insights. For instance, measure developers should be trained on integrat­ing AI tools that proactively identify and correct biases in health outcomes data, thereby crafting measures that genuinely reflect equitable health care practices. Health care providers, on the other hand, must understand how to responsibly apply these quality measures in a clinical context to ensure that AI-generat­ed insights enhance patient care across diverse populations.

Implementing Systematic Monitoring and Evaluation of AI Models in Health Care Quality Measurement

The successful integration of AI in health care quality measure­ment hinges not only on the initial design and training of AI models, but also on the ongoing monitoring and evaluation of their performance post-deployment. To ensure these models function as intended without exacerbating existing disparities or introducing new biases, a structured framework for continuous oversight is essential. This strategy involves regular assessments of AI-driven measurement, systematic detection of any biases, and implementation of corrective measures when necessary.

Key stakeholders in this process include AI and measure developers, health care administrators, clinicians, and regula­tory bodies. Notable regulatory bodies include the US Food and Drug Administration (FDA), which oversees the safety and effectiveness of AI applications in health care,^₂₁ CMS, and the Agency for Healthcare Research and Quality (AHRQ). Additionally, AI associations, such as the Association for the Advancement of Artificial Intelligence (AAAI), and quality organizations, such as the National Committee for Quality Assurance (NCQA), are instrumental in establishing standards and guidelines for AI in the health care sector. AI and measure developers are responsible for incorporating feedback mecha­nisms into AI and measurement systems that can constantly gather and analyze performance data. Health care administra­tors and clinician leaders must ensure AI systems are seamlessly integrated into clinical workflows and promote a culture in which measure users are encouraged to report discrepancies and anomalies in AI-assisted outcomes. Furthermore, regula­tory bodies must establish and enforce standards for AI per­formance, including adherence to ethical guidelines for AI in health care and mandatory reporting of AI evaluation results.

The monitoring and evaluation process can be operational­ized through several practical steps. First, AI tools should be deployed with established baseline metrics to regularly compare against AI performance. This involves historical data analysis and using automated, real-time analytics to help detect poten­tial biases early and trigger alerts for human oversight, thereby providing opportunities for course correction. Second, periodic reevaluation can determine whether AI is performing as expect­ed across various patient demographics and conditions. Third, it is crucial to implement a feedback loop in which insights from monitoring and evaluation are used to refine AI models.

The transformative potential of AI in enhancing quality measurement is undeniable. However, integrating AI into health care systems is not without significant challenges. Developing AI use standards, careful implementation, robust training efforts, ongoing evaluation, and continuous dialogue among key stakeholders can help ensure that AI tools do not inadvertently lead to unintended consequences or perpetuate biases. Research and experience at the intersection of AI and health care quality measurement are ongoing and evolving. The path ahead will require a balanced approach that embraces AI’s possibilities while vigilantly addressing its ethical, technical, and practical challenges. By committing to unbiased measure­ment and continuous improvement, the health care industry can harness the power of AI to transform quality measurement in ways that are both innovative and just, promoting better health outcomes for all patients.

Affiliations:

¹Real Chemistry, New York, NY

Correspondence:

Palak Patel, MHA

199 Water St 12th floor

New York, NY 10038

Phone: (1) 240-646-2622

Email: ppatel@realchemistry.com

Disclosures:

M.D., P.P., and T.V. are employed as consultants by Real Chemistry, an independent provider of analytics-driven, digital-first research, marketing services, and communications to the health care sector.