Abstract:Density estimation in high-dimensional settings is an important and challenging statistical this http URL methods based on kernel smoothing are inefficient in high dimensions due to the difficulties in specifying appropriate location-adaptive kernels. In this work, we introduce pre-training, a key idea behind many cutting-edge AI technologies, to the context of non-parametric density estimation. By establishing a pre-trained neural network that can recommend an appropriate location-adaptive kernel for each sample point, efficient density estimation with adaptive kernels is achieved in high dimensions. A wide range of numerical experiments show that this strategy is highly effective for improving density-estimation accuracy, when the target distribution is close to the distribution family for pre-training. When the target distribution is substantially different from the pre-training distribution family, the benefit from the proposed pre-training strategy may be diluted, but can be reactivated by an additional fine-tuning procedure.
| Comments: | 8 pages main text, 14 pages total including references and appendix, 3 figures |
| Subjects: | Machine Learning (stat.ML); Machine Learning (cs.LG); Methodology (stat.ME) |
| Cite as: | arXiv:2605.13092 [stat.ML] |
| (or arXiv:2605.13092v1 [stat.ML] for this version) | |
| https://doi.org/10.48550/arXiv.2605.13092 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Ruitong Zhang [view email]
[v1]
Wed, 13 May 2026 07:03:54 UTC (5,366 KB)