Study Flags Risk of Selection Bias in Evaluating Medicaid Health Plans
A new cross-sectional study published in JAMA Internal Medicine highlights the risk of selection bias when comparing Medicaid-managed care plans based on racial disparities in health care use and spending. The findings raise caution around using disparity metrics for public reporting or financial incentives without accounting for enrollee selection effects.
The study analyzed administrative claims and enrollment data from over 118 000 Black and White Medicaid enrollees in a southern US state from 2011 to 2015. Following the state’s 2012 transition to Medicaid managed care, most enrollees were randomly assigned to 1 of 5 health plans. The remainder selected their plan.
Researchers compared 4 utilization and spending outcomes—primary care visits, low-acuity emergency department visits, prescription drug fills, and total spending—between Black and White enrollees within each plan. Analyses were stratified by random vs self-selected plan assignment.
In the randomized population (70.2% of the sample), racial disparities in outcomes were substantial within plans but showed minimal variation across plans. This suggests that the plans themselves had little effect on these disparities. “Differences in racial disparities across plans were not statistically significant in the randomized population,” the authors wrote.
In contrast, the observational (self-selected) population showed marked variation across plans in the magnitude of racial differences for most outcomes. This variation remained statistically significant after adjustment for multiple comparisons, except for emergency department visits. The authors noted that “greater between-plan variation in racial differences in the observational population was only partially explained by sampling error.”
Crucially, the disparity estimates from the observational population diverged meaningfully from those observed under random assignment. Stratifying by race did not reconcile this gap, suggesting residual confounding. The results indicate that selection effects—ie, differences in who chooses which plan—can distort apparent disparities across plans.
These findings have implications for health equity initiatives that tie payment or performance assessments to disparity metrics. “Selection bias may mischaracterize plans’ relative performance on measures of health care disparities,” the authors concluded.
The study underscores the need for caution when using observational data to compare plans on equity performance. Without rigorous controls for selection effects, public reporting or payment adjustments may misrepresent true plan-level differences and risk unintended consequences for Medicaid enrollees and the plans that serve them.
Reference
Wallace J, Ndumele CD, Lollo A, et al. Attributing racial differences in care to health plan performance or selection. JAMA Intern Med. 2025;185(1):61–72. doi:10.1001/jamainternmed.2024.5451
