Two trends, opposite directions
Large language models are increasingly used to screen patients for randomized trials, and early systems do well on benchmarks and give clinician-readable reasons. A parallel literature documents sociodemographic bias in the same models. If trial automation inherits that bias, it could quietly distort who gets access to experimental therapies.
Same trial, different identity
The team took 58 US adult Phase II and III protocols registered on ClinicalTrials.gov in 2023 and 2024, led by oncology (13), infectious diseases (12) and neurology and pain (7). For each, 15 strictly clinical vignettes were written, five clearly eligible, five clearly ineligible and five borderline, and two board-certified physicians validated them.
Every vignette ran as a control with no identity and in 33 variants that differed only in the label: gender, race or ethnicity, socioeconomic status (high income, low income, unemployed, homeless) and sexual orientation. Nine models answered 20 screening questions across five domains, eligibility, risk-benefit, adherence, resources, and trust and attitude, for 5,324,400 evaluations. Differences under 0.10 on the 1 to 5 scale were treated as trivially small.
eligibility shift for homelessness, the only label past the trivial band
the same label on resource sufficiency, about six times larger
resources for high-income cases, the largest shift upward
Within the rules, stable
Eligibility judgments barely moved. Most adjusted differences fell within 0.05 of the control on the 1 to 5 scale, and race and ethnicity alone were essentially zero once socioeconomic status was accounted for: Black +0.008, Asian -0.004, White +0.024. The one label that crossed the trivial threshold was homelessness, at -0.121 (95% CI -0.15 to -0.09; P < .001).
Each label's difference from the control vignette on the 1 to 5 scale, nine models pooled; crimson below, where the label lowered the score, blue above. The gold band marks differences under 0.10, treated as trivially small. Hover or tap a row to read it. Data from Supplement Table 2 of the preprint, with 95% confidence intervals where its text gives them; two transgender rows are omitted because the preprint's text and table disagree on them.
Beyond the rules, disparity
The secondary domains told a different story. Resources showed the widest socioeconomic spread: homeless -0.715, Black homeless -0.689, low income -0.207, high income +0.129. Adherence followed the same gradient: homeless -0.595, Black homeless -0.542, low income -0.138, high income +0.087. In trust and attitude, Black homeless (-0.372) and homeless (-0.337) fell furthest.
Risk-benefit, the most rule-bound of the secondary domains, behaved like eligibility: every label stayed between -0.086 and +0.023. Domains that apply explicit criteria to fixed clinical content held steady; domains that invite inference about behavior or capacity moved with socioeconomic cues.
Within fixed criteria the models reasoned consistently. Outside them, they echoed the inequities of their data.
Homeless: eligibility -0.121, risk-benefit -0.086, trust -0.337, adherence -0.595, resources -0.715. Past the 0.10 band in 4 of 5 domains.
The five domains on one scale. Each gray mark is one of the 31 labels, placed by its difference from the control vignette on the 1 to 5 scale, nine models pooled; the tiles follow one label, crimson where it lowered the score and blue where it raised it. The gold band marks differences under 0.10. Hover or tap a mark to read it, and tap to follow that label. Data from Supplement Table 2 of the preprint.
The safeguard is separation
The risk sits at the edge of the eligibility logic. If soft judgments about adherence, resources or trust feed recruitment or resource decisions, disparities can re-enter through pathways other than the inclusion criteria. The authors' safeguard is separation: keep eligibility tethered to protocol language, and treat the other domains as planning variables, never as filters.
The study covered US adult Phase II and III trials from 2023 and 2024, used simplified socioeconomic proxies and standardized vignettes, and tested each model at one point in time with default settings.
Bias in model-assisted trial screening is conditional, and it lives outside the rules.
Across 5.3 million evaluations from nine models on 58 protocols, eligibility judgments held steady across 33 identity labels. Once the questions moved past explicit criteria, homelessness, low income and unemployment pulled adherence, resources and trust down. Responsible deployment depends on keeping eligibility tied to the protocol.
Shelly Soffer, Mahmud Omar, Orly Efros, Donald U. Apakama, Aya Mudrik, Robert Freeman, Girish N. Nadkarni, Eyal Klang
Rabin Medical Center, Tel Aviv University, Icahn School of Medicine at Mount Sinai, Sheba Medical Center, Ben-Gurion University of the Negev
Journal of the American Medical Informatics Association 33(8), 1504-1509 (2026)