Abstract:We introduce and analyze Target-Induced Loss Tilting (TILT) for unsupervised domain adaptation under covariate shift. It is based on a novel objective function that decomposes the source predictor as $f+b$, fits $f+b$ on labeled source data while simultaneously penalizing the auxiliary component $b$ on unlabeled target inputs. The resulting fit $f$ is deployed as the final target predictor. At the population level, we show that this target-side penalty implicitly induces relative importance weighting at the population level, but in terms of an estimand $b^*_f$ that is self-localized to the current error, and remains uniformly bounded for any source-target pair (even those with disjoint supports). We prove a general finite-sample oracle inequality on the excess risk, and use it to give an end-to-end guarantee for training with sparse ReLU networks. Experiments on controlled regression problems and shifted CIFAR-100 distillation show that TILT improves target-domain performance over source-only training, exact importance weighting, and relative density-ratio baselines, with a stable dependence on the regularization parameter.
| Comments: | 32 pages, 17 figures. Submitted to NeurIPS 2026 |
| Subjects: | Machine Learning (cs.LG); Machine Learning (stat.ML) |
| MSC classes: | 62G08, 62G20, 68T07 |
| Cite as: | arXiv:2605.14280 [cs.LG] |
| (or arXiv:2605.14280v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2605.14280 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Kakei Yamamoto [view email]
[v1]
Thu, 14 May 2026 02:26:34 UTC (5,529 KB)