Abstract:Diffusion models are often trained in low-dimensional latent spaces, which are then reused for related but shifted datasets. In this work, we study when such latent reuse remains reliable under distribution shift. We consider a source-target setting in which both datasets are approximately low-dimensional but may lie near different subspaces. We show that freezing and reusing a source latent space induces a target-domain score error governed by two quantities: the principal-angle misalignment between the source and target subspaces, and the target ambient noise amplified by the diffusion time scale. Motivated by these limits, we further study mixed source-target training and characterize how the required shared latent dimension depends on the relative geometry of the two distributions. Our results provide theoretical guidance on when latent reuse is reliable and when learning a shared representation may be necessary.
| Subjects: | Machine Learning (stat.ML); Machine Learning (cs.LG); Probability (math.PR) |
| Cite as: | arXiv:2605.13448 [stat.ML] |
| (or arXiv:2605.13448v1 [stat.ML] for this version) | |
| https://doi.org/10.48550/arXiv.2605.13448 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Yifeng Yu [view email]
[v1]
Wed, 13 May 2026 12:42:58 UTC (37 KB)