Frontier-level reasoning from a 1.7B model.
We trained Qwen3 1.7B for a specific radiology workflow and measured it against a radiologist-validated benchmark.
0.324 → 0.982 on our radiologist-validated benchmark.How far can a 1.7B model go on a serious clinical reasoning task?
What we tested and what changed.Test whether Qwen3 1.7B could learn stronger clinical reasoning for a specific radiology workflow, without calling a frontier model at inference.
Train on clinician-reviewed reasoning traces, then preference-tune against the model's own convincing mistakes. Expert radiologists anchored the evaluation.
Across 833 held-out answers, mean reasoning correctness rose from 0.324 to 0.982: a 203% improvement with less than $50 spent on training and candidate-data generation.
The gap between capability and cost.
Frontier models can handle clinical reasoning, but using them for every request raises deployment and operating costs.
Frontier LLM
Capable across many tasks, with significant recurring costs.
- Data movement can complicate private deployment
- Token usage and latency recur on every call
Small off-the-shelf model
Less expensive to run privately, but weak on clinical reasoning without adaptation.
- Limited task-specific reasoning
- Plausible answers can conceal incorrect logic
A narrow workflow creates a testable middle ground: train a smaller model for the task and measure whether it is reliable enough to deploy.
We made it deliberately severe.
The task was radiology: given clinical context and findings, answer specific clinical questions with a structured reasoning trace and a final answer.
Qwen3 1.7B can run locally or on a department server while retaining enough capacity to learn from domain data.
Including the frontier-model calls used to generate candidate reasoning traces. The whole loop had to fit the budget.
The scope was intentionally narrow: one clinically meaningful workflow with a defined evaluation set.
How we trained and tested it.
The work covered the data pipeline through final evaluation, with expert review at the points where errors mattered most.
Curate data
Convert anonymized radiology cases into training examples with expert-verified reasoning traces.
Clinician quality gateSupervised fine-tune
Teach the model to connect case findings and prior context to an actionable conclusion.
SFTPreference-tune
Pair flawed model outputs with expert-verified answers so the model learns which reasoning to prefer.
Preference tuningEvaluate
Score held-out answers with a judge calibrated against ratings from three expert radiologists.
Radiologist-anchoredFrontier models generated candidate reasoning traces within the $50 budget. Clinicians filtered and corrected those candidates before training. Preference tuning then used the model's own incorrect outputs as rejected examples.
Results on held-out questions.
Three expert radiologists scored a sample using the clinical rubric. We calibrated an LLM judge against their ratings, then applied the audited judge to all 833 held-out answers.
Reasoning correctness +203%
Mean score on the raw 0–2 clinical rubric, before and after training. Bars are drawn to the true 0–2 scale.
The distribution of answers shifted out of the risky band.
The extended rubric grouped all 833 answers by clinical quality. Weak or unsafe answers fell from 383 to 201. Strong answers increased from 97 to 264.
Expert review also found better grounding and stronger agreement between the evidence cited and the final conclusion.
Scope: These results come from an internal benchmark, not a clinical clearance study. Radiologist scoring was used to validate the evaluation method.
Route the right work to the right model.
A benchmark can identify tasks that a smaller private model can handle reliably, reducing dependence on frontier-model calls.
Frontier intelligence
- Planning and ambiguous requests
- Escalation
The same method can be tested on other bounded workflows. Keep complex requests with a frontier model; route a task to a smaller model only when its benchmark supports that decision.
Which parts of your workflow require a frontier model?
Talk to us about evaluating where a smaller private model could reduce cost without lowering the quality bar.