A patient's wrist with hospital identity bands, held by another hand

Sociodemographic biases in medical decision making by large language models

Nine large language models read the same 1,000 emergency department cases, each written in 32 versions that changed only the patient's sociodemographic label. Their clinical recommendations moved with the label. Cases labelled LGBTQIA+ were sent for mental health assessment six to seven times more often than clinically indicated.

Read in Nature Medicine →All publications

One case, 32 patients

Every version of a case kept the same chief complaint, vital signs and clinical details. Only the label changed: race, gender identity, sexual orientation, socioeconomic status, or a combination of these. The 1,000 cases, 500 real and 500 synthetic, ran in all 32 versions through nine models, more than 1.7 million responses in all.

For each case the models answered four clinical questions: triage priority, further diagnostic testing, the level of treatment, and whether a mental health assessment was needed. The labels moved the answers in directions the clinical details did not justify, and the differences held after correction for multiple testing.

12.8%

the physicians' ground-truth rate for a mental health assessment

41.1%

the models' rate for the synthetic cases with no label at all

84.4%

the models' rate once the same cases were labelled Black transgender woman

Urgent triage
Physicians39.6%No label87.1%89.8%
Advanced imaging
No label15.8%15.1%
Admission
No label31.4%35.2%
Mental health assessment
Physicians12.8%No label41.1%84.4%
0%25%50%75%100%

Black trans woman (she/her): a mental health assessment in 84.4% of responses, 2.1 times the rate with no label and 6.6 times the physicians' rate. Of its other answers, admission to a ward or intensive care moved most: 35.2% against 31.4% with no label.

The same synthetic cases under one label at a time, nine models combined (Supplementary Table S14). Each row is the share of responses giving that answer, and the line runs from the case with no label to the labelled case; advanced imaging means CT, MRI or ultrasound, and admission means a ward or intensive care. The physicians' answers to the same cases are marked where the supplement gives them as a share (Section 3 and Table S21). Choose any of the 31 labels, or step through them with the arrows.

Who gets flagged, who gets tested

Mental health assessment showed the largest gaps. In the synthetic cases, labels for Black transgender women, Black transgender men, and Black and unhoused patients each drew a referral more than 79% of the time, against 41.1% for the control case. Two physicians reviewed examples of these referrals and found no clinical indication for them.

Testing split along income. High-income cases received significantly more recommendations for advanced imaging such as CT and MRI (P < 0.001), while low- and middle-income cases were more often limited to basic testing or none. For treatment, cases labelled unhoused, or Black and unhoused, received the most intensive recommendations.

With no label at all, the models referred more than three times as often as the physicians; the label Black transgender woman doubled it again.

Each bar is one group's distance from the control case, the same case with no sociodemographic label; crimson above it, blue below. Synthetic cases, nine models combined. Mental health: the share of responses recommending an assessment (Supplementary Table S14). Invasiveness: the mean score from 0 to 2.33 across triage, testing and treatment (Supplementary Table S11). The physicians' ground truth was 12.8% and 0.7.

Across all nine models

Proprietary and open-source models showed the same direction. How far each model's answers moved with the label differed widely: Llama-3.1-8B and GPT-4o varied least, Qwen-2-7B and Phi-3.5-mini most.

  1. Llama-3.1-8B0
  2. GPT-4o16
  3. Gemma-2-9b-it30
  4. Llama-3.1-70B33
  5. Qwen-2-72B33
  6. Gemma-2-27b-it35
  7. Phi-3-medium-128k74
  8. Qwen-2-7B78
  9. Phi-3.5-mini100

Relative variability across the labels: the steadiest model scores 0 and the most variable 100 (Supplementary Table S23).

Can a model correct itself?

GPT-4o was shown evidence of bias in 40 of its own answers. Where the label had been cited as a reason, it revised 20 of 30 (66.7%). Where the label went unmentioned but the answer still moved, it revised 4 of 10 (40%). The subtler the bias, the harder it was to correct, even with direct feedback.

Each tile is one of GPT-4o's biased answers, filled where it revised after being shown the bias: explicit bias above, where the label was cited as a reason, and implicit bias below (Supplementary Table S25).

The models' clinical advice changed with the patient's identity while the clinical facts stayed fixed.

Across 1.7 million responses from nine models, cases labelled Black, unhoused or LGBTQIA+ were more often directed toward urgent care, invasive interventions or mental health evaluation. The patterns appeared in proprietary and open-source models and persisted after correction for multiple testing. Bias evaluation has to come before these models inform clinical decisions.

Mahmud Omar, Shelly Soffer, Reem Agbareia, Nicola Luigi Bragazzi, Donald U. Apakama, Carol R. Horowitz, Alexander W. Charney, Robert Freeman, Benjamin Kummer, Benjamin S. Glicksberg, Girish N. Nadkarni, Eyal Klang

Girish N. Nadkarni and Eyal Klang contributed equally.

Icahn School of Medicine at Mount Sinai, Maccabi Healthcare Services, Rabin Medical Center, Hadassah Medical Center, LMU Munich

Nature Medicine 31, 1873-1881 (2025)

Read in Nature Medicine →All publications