Large language models being developed to guide pain management recommendations exhibit troubling racial and demographic biases that could worsen existing health disparities, according to new research published in Nature Health.
Researchers tested ten AI language models on 1,000 acute pain scenarios, presenting each case with 34 different socio-demographic variations. The results revealed stark inconsistencies that particularly affected historically marginalized communities.
Black patients, unhoused individuals and those identifying as LGBTQIA+ often received more or stronger opioid recommendations from the AI systems, with some exceeding 90% in cancer cases. Paradoxically, these same models simultaneously flagged these groups as high risk for substance abuse.
Meanwhile, low-income or unemployed patients faced the opposite problem. Despite being assigned elevated risk scores, they received fewer opioid recommendations, suggesting contradictory reasoning embedded in the AI systems.
Disparities in anxiety treatment recommendations and perceived psychological stress similarly clustered around marginalized populations, even when clinical details remained identical across patient scenarios. The patterns diverged significantly from standard medical guidelines.
The findings point to model-driven bias rather than legitimate clinical variation. Such biases could have serious real-world consequences given the ongoing opioid epidemic and the critical need to balance effective pain management with addiction risks.
The research analyzed 3.4 million AI-generated responses overall, underscoring the scale at which these biases could affect patient care if left unaddressed. Researchers emphasized the urgent need for rigorous bias evaluation and integration of guideline-based checks in AI medical tools to ensure equitable, evidence-based pain care across all demographic groups.
See: “Socio-demographic gaps in pain management guided by large language models” (February 6, 2026)
