Abstract:Tensor neural networks (TNNs) have demonstrated their superiority in solving high-dimensional problems. However, similar to conventional neural networks, TNNs are also influenced by the Frequency Principle, which limits their ability to accurately capture high-frequency features of the solution. In this work, we analyze the training dynamics of TNNs by Fourier analysis and enhance their expressivity for high-dimensional multi-scale problems by incorporating random Fourier features. Leveraging the inherent tensor structure of TNNs, we further propose a novel approach to extract frequency features of high-dimensional functions by performing the Discrete Fourier Transform to one-dimensional component functions. This strategy effectively mitigates the curse of dimensionality. Building on this idea, we propose a frequency-adaptive TNNs algorithm, which significantly improves the ability of TNNs in solving complex multi-scale problems. Extensive numerical experiments are performed to validate the effectiveness and robustness of the proposed frequency-adaptive TNNs algorithm.
| Subjects: | Machine Learning (cs.LG); Mathematical Physics (math-ph) |
| Cite as: | arXiv:2508.15198 [cs.LG] |
| (or arXiv:2508.15198v2 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2508.15198 arXiv-issued DOI via DataCite |
Submission history
From: RuKang You [view email]
[v1]
Thu, 21 Aug 2025 03:16:52 UTC (585 KB)
[v2]
Thu, 14 May 2026 10:18:52 UTC (585 KB)