One agent cannot do it all
Large language models already retrieve literature, extract data from notes and compute drug doses. Real clinical systems do not send tasks one at a time; a busy hospital may need dozens handled together. This study asked what happens to accuracy as the workload grows.
Two designs were tested head to head. A single agent received the whole batch, with access to every tool. In the orchestrated design, a lightweight controller dispatched each task to a dedicated worker with one tool: a k-NN retriever over 234,650 PubMed oncology records, a note fetcher over 331,793 MIMIC-IV discharge summaries, or a sandboxed calculator for 20 dosing templates. Batches held 5, 10, 20, 40 or 80 mixed tasks, ten batches per setting, across four models.
pooled accuracy of the orchestrated design at 80 tasks, from 90.6% at 5
pooled accuracy of the single agent at 80 tasks, from 73.1% at 5
fewer tokens for GPT-4.1-mini at 80 tasks: about 60,000 against 3.9 million
Accuracy diverges at scale
With five tasks, the single agent scored 73.1% and the orchestrated system 90.6%. As batches grew the gap widened, and every difference beyond ten tasks was significant (Welch t-test, FDR-adjusted P < 0.01). GPT-4.1-mini showed the starkest contrast: its single agent fell from 96.0% to 33.9%, while its orchestrated system stayed at 91% or above.
The models did not all behave alike. GPT-4.1-nano's single agent fell to 10.4% at 80 tasks. Qwen-3-8B's single agent was weak from the start, and its orchestrated system also dropped, to 29.0% at 80 tasks. Llama-2-70B's single agent held up to ten tasks, and its largest batches failed outright, returning no valid JSON.
Each worker sees one task, one tool and a short prompt. The single agent sees everything at once.
GPT-4.1-mini, 80 tasks: orchestrated 91.4%, single agent 33.9%.
Mean accuracy over ten batches at each batch size, orchestrated (blue) and single agent (crimson). Tap or hover a batch size for its values. Data from Supplementary Table S1; Llama-2-70B's single agent has no value at 80 tasks because its runs failed.
Why orchestration works
The paper points to context overload. In the single-agent setup, dozens of unrelated tasks share one context window, consistent with the documented lost-in-the-middle effect. The orchestrator constrains each worker to a single tool and a short prompt, and gathers the results.
Token efficiency
The orchestrated runs were far cheaper. For GPT-4.1-mini at 80 tasks, an orchestrated batch used about 60,000 tokens and a single-agent batch about 3.9 million, a 65-fold difference; for GPT-4.1-nano the gap was about 17-fold (53,209 against 891,986). Token comparisons for Llama and Qwen are confounded by their early failures.
GPT-4.1-mini, 80 tasks: orchestrated 60,371 tokens per batch, single agent 3,872,443, 64 times as many.
Mean tokens per batch over ten batches, on a log scale, orchestrated (blue) and single agent (crimson); the numbers along the top are how many times as many tokens the single agent used. Tap or hover a batch size for its values. Data from Supplementary Table S3. The paper's 65-fold divides the rounded 3.9 million by 60,000. Llama-2-70B and Qwen-3-8B are left out, because their early failures confound the counts.
Divide the work and the accuracy holds.
An orchestrator that routes each task to a dedicated worker kept accuracy higher and token use far lower as clinical workloads grew, while a single agent collapsed under the same load. Task-level delegation is a practical way to run current models on mixed clinical work at scale.
Eyal Klang, Mahmud Omar, Ganesh Raut, Reem Agbareia, Prem Timsina, Robert Freeman, Nicholas Gavin, Lisa Stump, Alexander W. Charney, Benjamin S. Glicksberg, Girish N. Nadkarni
Icahn School of Medicine at Mount Sinai, Mount Sinai Medical Center, Hadassah Medical Center
npj Health Systems 3, 23 (2026)