Abstract:Neural population models, which predict the joint firing of many simultaneously recorded neurons forward in time, are typically evaluated by a single aggregate Pearson correlation $r$ between predicted and actual spike counts, a number that masks critical structure. We argue that how we evaluate spike forecasting matters as much as what we build, and introduce SpikeProphecy, the first large-scale benchmark for causal, autoregressive spike-count forecasting on real electrophysiology recordings. Our core contribution is a population metric decomposition that separates aggregate performance into temporal fidelity, spatial pattern accuracy, and magnitude-invariant alignment. The decomposition surfaces aspects of the underlying data that an aggregate scalar collapses together. We apply the protocol to 105 Neuropixels sessions (Steinmetz 2019 + IBL Repeated Site; ~89,800 neurons) with seven architecture baselines spanning four structural families: four SSMs (three diagonal and one non-diagonal), a Transformer, an LSTM, and a spiking network. The decomposition surfaces a brain-region predictability ranking that reproduces across all seven baselines and survives ANCOVA correction for firing-statistics constraints (region $\Delta R^2 = 0.018$ above the firing-statistics covariates). It also exposes a sub-Poisson evaluation floor where rigorous metrics combine with genuine biophysical constraints on regular spike trains, and yields a negative result on KL-on-output-rates distillation for ANN-to-SNN transfer in this Poisson count domain.
| Comments: | 26 pages, 4 figures, 12 tables; submitted to NeurIPS 2026 Datasets and Benchmarks Track; processed dataset at this https URL (CC-BY-4.0); code at this https URL |
| Subjects: | Neurons and Cognition (q-bio.NC); Machine Learning (cs.LG) |
| Cite as: | arXiv:2605.12992 [q-bio.NC] |
| (or arXiv:2605.12992v1 [q-bio.NC] for this version) | |
| https://doi.org/10.48550/arXiv.2605.12992 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: John Minnick R [view email]
[v1]
Wed, 13 May 2026 04:45:35 UTC (1,040 KB)