Abstract:Mixed-effects models are widely used to model data with hierarchical grouping structures and high-cardinality categorical predictor variables. However, for high-dimensional crossed random effects, current standard computations relying on Cholesky decompositions can become prohibitively slow. In this work, we present Krylov subspace-based methods that address existing computational bottlenecks, and we analyze them both theoretically and empirically. In particular, we derive new results on the convergence and accuracy of the preconditioned stochastic Lanczos quadrature and conjugate gradient methods for mixed-effects models, and we develop scalable methods for calculating predictive variances. In experiments with simulated and real-world data, the proposed methods yield speedups by factors of up to about 10,000 and are numerically more stable than Cholesky-based computations.
| Subjects: | Methodology (stat.ME); Machine Learning (cs.LG); Machine Learning (stat.ML) |
| Cite as: | arXiv:2505.09552 [stat.ME] |
| (or arXiv:2505.09552v3 [stat.ME] for this version) | |
| https://doi.org/10.48550/arXiv.2505.09552 arXiv-issued DOI via DataCite |
Submission history
From: Pascal Kündig [view email]
[v1]
Wed, 14 May 2025 16:50:19 UTC (1,577 KB)
[v2]
Wed, 17 Dec 2025 20:11:18 UTC (4,148 KB)
[v3]
Thu, 14 May 2026 06:48:00 UTC (4,184 KB)