An open hand holding three capsules in a shaft of light

Socio-demographic gaps in pain management guided by large language models

Ten large language models read 1,000 acute-pain cases, half cancer and half not, each in 34 sociodemographic versions. Across 3.4 million responses, cases labelled unhoused, Black or LGBTQIA+ often received more or stronger opioid recommendations, while the same models rated them as higher risk.

Read in Nature Health →All publications

Same pain, different prescriptions

Pain is among the most common reasons people come to an emergency department, and large language models are being positioned to help with the treatment decision. The 1,000 physician-validated vignettes were split evenly between cancer and non-cancer settings. Each ran as a control with no identifier and in 33 labelled versions, through ten open and closed models.

The label changed the prescription. In non-cancer cases, Black unhoused patients had the highest odds of an opioid recommendation (OR 1.73, P < 0.001), followed by unhoused patients with no racial identifier (OR 1.64) and White unhoused patients (OR 1.61). Low income (OR 0.78) and middle income (OR 0.72) lowered the odds below the control.

OR 1.73

for an opioid recommendation when a non-cancer case was labelled Black and unhoused

84.8%

of cancer cases labelled Black and unhoused got an opioid, against 79.5% with no label

OR 8.35

for flagging psychological stress in the same label, in non-cancer cases

High risk, high prescribing

The models contradicted themselves. In non-cancer cases they gave Black unhoused and White unhoused patients risk scores of 4.02 and 4.22 for misuse, against 2.84 for the control, and still recommended them more opioids; in cancer settings, opioid rates for marginalized groups sometimes exceeded 90%. Low-income and unemployed patients went the other way: higher risk scores, fewer opioids, more monitoring.

The groups the models rated highest risk for misuse were the groups they gave the most opioids.

Each bar is one group's opioid recommendation rate minus the control's, ten models pooled; crimson above the control, blue below. The rate is the sum of the mild, moderate and high opioid categories in Supplementary Table S4. Differences under about one point are mostly not significant in the paper's mixed-effects models.

Mental health and stress flags

The labels also drew psychological flags. Anxiety treatment was recommended most for Black unhoused and unhoused patients (both OR 1.48), then for White unhoused patients and Black transgender women (both OR 1.26) and low-income patients (OR 1.23). In non-cancer cases, the models judged psychological stress to be shaping the pain most often for Black unhoused patients (OR 8.35), unhoused patients with no racial identifier (OR 7.45) and White unhoused patients (OR 5.51).

  1. Black unhoused8.35
  2. Unhoused7.45
  3. White unhoused5.51
  4. Black trans woman (she/her)3.51
  5. Black gay/lesbian2.69
  6. Low income2.51
  7. Black trans man (he/him)2.36
  8. Unemployed2.19
  9. White0.59

Odds that the models judged psychological stress to be shaping the pain, against the case with no label, in non-cancer cases; the dark mark on each line is an odds ratio of 1, no difference. Mixed-effects models in the preprint, all P < 0.001.

Flags like these risk recasting a physical complaint as a psychological one, and pulling clinical attention away from the pain itself.

The cancer label

A cancer diagnosis raised prescribing for everyone: the control's opioid rate was 37.7% in non-cancer cases and 79.5% in cancer cases. It did not remove the gaps. In non-cancer cases, opioid rates ran from 35.7% for middle income to 41.0% for Black unhoused patients; in cancer cases, from 77.2% for non-binary patients to 84.8% for Black unhoused patients.

No medicationNon-medicated reliefLow-dose NSAIDsModerate NSAIDsHigh-dose NSAIDsMild opioidsModerate opioidsHigh opioids
No label37.7%opioids
Black unhoused41.0%opioids

Black unhoused, non-cancer cases: an opioid in 41.0% of responses, against 37.7% with no label. The largest change was in moderate opioids: 24.9% against 22.7%.

Every recommendation the ten models made for the same cases, pooled, from no medication on the left to high opioids on the right; the crimson tiles are opioids. Tap or hover a tile for its share. Data from Supplementary Table S4.

Pain care advice from these models changed with the patient's label while the pain stayed the same.

Across 3.4 million responses, the models gave more opioids to some historically marginalized groups while rating the same groups as higher risk. The patterns diverged from standard guidelines and point to model-driven bias rather than acceptable clinical variation. Bias evaluation and guideline-based checks need to be in place before these models inform prescribing.

Mahmud Omar, Shelly Soffer, Reem Agbareia, Nicola Luigi Bragazzi, Benjamin S. Glicksberg, Yasmin L. Hurd, Donald U. Apakama, Alexander W. Charney, David L. Reich, Girish N. Nadkarni, Eyal Klang

Icahn School of Medicine at Mount Sinai, Rabin Medical Center, Hadassah Medical Center, University of Parma

Nature Health 1(2), 216-225 (2026)

Read in Nature Health →All publications