FIELD NOTES / CLINICAL AI
AI is learning clinical judgment by practicing on simulated patients.
ResidencyRL trained a model across complete virtual consultations instead of isolated medical questions. It asked better questions, missed fewer red flags, and earned a strong expert preference. The same study also shows why this is not yet an autonomous doctor.
Up to 60 dialogue turns and 8 tool calls in one training trajectory.
01 / THE PROBLEM
Knowing medicine is not the same as conducting a consultation.
A medical exam gives the model a question and the facts needed to answer it. A real consultation does not. The clinician must decide what to ask next, notice what is missing, revise a diagnosis, choose a safe action, explain it, and know when to escalate.
That makes clinical reasoning a sequence problem. One early assumption can change every later question. One missed red flag can make an otherwise fluent answer unsafe.
ResidencyRL, submitted on August 7, tests whether reinforcement learning across complete simulated encounters can improve this process. The starting point was Gemini 3.5 Flash. The training environment made it practice, receive structured feedback, and try again.
The relevant information is already present.
The next useful fact must be discovered.
02 / THE SIMULATED RESIDENCY
Practice happens inside a generated clinical world.
The model is rewarded for the quality of the whole encounter, not only its final diagnosis.
- 01
Generate a case
A scenario defines the condition, history, required questions, red flags, management, and safety constraints.
- 02
Simulate the patient
An LLM patient reveals information through dialogue and can use complex or adversarial behavior.
- 03
Run the encounter
The policy model asks questions, updates its view, uses tools, proposes management, and writes documentation.
- 04
Score the trajectory
Structured judges evaluate diagnosis, management, intake, communication, documentation, style, and safety.
- 05
Reinforce better behavior
The training update makes stronger complete trajectories more likely in the next round.
Most cases taught routine breadth. Smaller sets targeted process and danger.
03 / WHAT IMPROVED
The strongest gain was a better search for missing information.
Held-out adversarial encounters tested whether the model would close too early on a diagnosis or overlook a safety signal.
The gain and the residue must be read together. A 31% relative reduction in missed red flags is meaningful. A 31.5% remaining miss rate in these difficult simulations is also too high for an autonomy claim.
Management reasoning
On 120 AMIE Mx scenarios, the score rose from 80.07% to 88.41%. Communication rose by 8.50 points and clinical skills by 5.26.
Oncology comparison wins
Across 300 unseen oncology cases, ResidencyRL won 42.9% of composite comparisons versus 18.6% for the baseline. The rest were ties.
Promising, not conclusive
AgentClinic and CRAFT-MD results generally moved in the favorable direction, but most individual gains were not statistically significant.
04 / WHAT THE EXPERT PREFERENCE MEANS
Clinicians saw a better consultation. They did not validate an AI replacement.
preferred for completeness of information gathering
preferred for management-plan appropriateness
preferred for diagnostic assessment
preferred or tied for management-plan safety
05 / THE THREE-MONTH FRONTIER
Medical AI is assembling a larger capability stack.
Only research published from May 12 through August 12, 2026 is used here.
Video closes one gap and reveals others.
An August 10 AMIE Video study combined low-latency dialogue, clinical planning, and audio-visual perception. Actor-based evaluation was strong, while fine anatomical precision, subtle affect, and fast movement remained difficult.
Longitudinal care needs memory and evidence.
A June Nature study tested management across three visits with persistent state and guideline retrieval. Its authors explicitly say the system is not ready for clinical care.
World models try to add consequences.
SepsisAgent proposes a treatment, simulates a predicted response, and refines the plan. It is an important direction, but retrospective off-policy value is not a real patient outcome.
06 / WHERE THE EVIDENCE STOPS
The evaluator, simulator, and real world are three different things.
Simulated patients
The environment is text-only, English, and oriented toward US telehealth. It does not include physical examination, delayed lab results, procedures, real provider coordination, or ordinary clinical disruption.
Evaluator bias
The paper finds that automated judges systematically favored the trained model more strongly than expert clinicians did. Training against a judge can improve the judge's score without closing every clinical gap.
No patient outcomes
The study measures behavior in simulations and actor-based review. It does not show lower morbidity, safer treatment, better adherence, or improved health in real patients.
Limited reproduction
Proprietary infrastructure and incomplete optimization details prevent an independent team from reproducing the full training result from the paper alone.
A model can only learn the situations its simulator can create and the qualities its evaluator can recognize.
Supervised, bounded assistance
- structured history collection
- clinician preparation
- guideline-grounded review
- documentation
- second-pass red-flag checks
Autonomous clinical care
- independent diagnosis
- unsupervised treatment
- safe real-world escalation
- better patient outcomes
- general performance across settings
THE BOTTOM LINE
This is evidence that practice can create capability. It is not evidence that simulation has solved medicine.
ResidencyRL makes a precise advance. It turns clinical reasoning from a collection of answers into a behavior that can be practiced across long sequences. The model became more complete, more cautious about red flags, and more useful than its baseline in blinded expert review.
The result also exposes the next bottleneck. Better simulation can create more practice, but only the real world can show whether the learned behavior improves care. Progress now depends on realistic environments, independent evaluation, prospective studies, and human accountability.
07 / RECENT SOURCES
Every current claim uses the latest three-month window.
Evidence cutoff: August 12, 2026. Window: May 12 through August 12, 2026. Preprints are labeled and interpreted more cautiously than peer-reviewed studies.
- ResidencyRL: Reinforcement Learning in Simulated Clinical EnvironmentsAug 7, 2026 · primary preprint
- Towards Expert-level Medical AI for Real-time Video ConsultationsAug 10, 2026 · primary preprint
- Toward a test of medical AI superintelligenceJul 27, 2026 · Nature Medicine perspective
- AI agents in clinical practice: an evidence mapJul 13, 2026 · npj Digital Medicine perspective
- Towards conversational artificial intelligence for disease managementJun 17, 2026; version of record Jul 22 · Nature
- Agents AMIE and MIRA advance medical AI capabilitiesJun 30, 2026 · Nature Medicine research highlight
- Towards autonomous medical artificial intelligence agentsJun 17, 2026 · Nature
- Medical world models: representing medical states, modelling clinical dynamics and guiding intervention policiesJun 15, 2026 · review preprint
- ClinicalMC: A Benchmark for Multi-Course Clinical Decision-MakingJun 2, 2026 · benchmark preprint
- Advancing conversational diagnostic AI with multimodal reasoningMay 14, 2026 · Nature Medicine
- Agentifying Patient Dynamics through a Clinical World ModelMay 14, 2026 · primary preprint