Abstract:Adapting language models to the clinical domain through continued pretraining and instruction tuning requires costly retraining for each new model generation. We propose Cross-Architecture Proxy Tuning (CAPT), a model-ensembling approach that enables training-free adaptation of state-of-the-art general-domain models using existing clinical models. CAPT supports models with disjoint vocabularies, leveraging contrastive decoding to selectively inject clinically relevant signals while preserving the general-domain model's reasoning and fluency. On six clinical classification and text-generation tasks, CAPT with a new-generation general-domain model and an older-generation clinical model consistently outperforms both models individually and state-of-the-art ensembling approaches (average +17.6\% over UniTE, +41.4\% over proxy tuning across tasks). Through token-level analysis and physician case studies, we demonstrate that CAPT amplifies clinically actionable language, reduces context errors, and increases clinical specificity. This technique especially benefits healthcare institutions with constrained computational capacity that cannot support iterative clinical training and want to adopt emerging general-domain model advances.
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2601.03423 [cs.CL] |
| (or arXiv:2601.03423v3 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2601.03423 arXiv-issued DOI via DataCite |
Submission history
From: Sasha Ronaghi [view email]
[v1]
Tue, 6 Jan 2026 21:23:47 UTC (5,646 KB)
[v2]
Mon, 19 Jan 2026 20:30:45 UTC (5,646 KB)
[v3]
Tue, 28 Apr 2026 22:36:58 UTC (6,422 KB)