Abstract:Validating training data for reasoning models typically requires expensive trial-and-error fine-tuning cycles. In this work, we investigate whether the utility of a reasoning dataset can be reliably predicted prior to training using intrinsic data metrics. We propose a suite of quantitative measures and evaluate their predictive power by fine-tuning 8B and 11B models on semantically distinct variants of a Polish reasoning dataset. Our analysis reveals that these intrinsic metrics demonstrate strong and significant correlations with downstream model performance. Crucially, we find that the predictors of utility are scale-dependent: smaller models rely on alignment-focused metrics to ensure precision, whereas larger models benefit from high redundancy, utilizing verbose traces to solve complex tasks. These findings establish a scale-aware framework for validating reasoning data, enabling practitioners to select effective training sets without the need for exhaustive empirical testing.
| Comments: | To appear in the Proceedings of the International Conference on Computational Science (ICCS) 2026 |
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2605.13290 [cs.AI] |
| (or arXiv:2605.13290v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2605.13290 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Dzmitry Pihulski [view email]
[v1]
Wed, 13 May 2026 10:04:38 UTC (146 KB)