Authors:Congmin Zheng, Jiachen Zhu, Zhuoying Ou, Yuxiang Chen, Kangning Zhang, Rong Shan, Zeyu Zheng, Mengyue Yang, Jianghao Lin, Yong Yu, Weinan Zhang
Abstract:Although Large Language Models (LLMs) exhibit advanced reasoning ability, conventional alignment remains largely dominated by outcome reward models (ORMs) that judge only final answers. Process Reward Models(PRMs) address this gap by evaluating and guiding reasoning at the step or trajectory level. This survey provides a systematic overview of PRMs through the full loop: how to generate process data, build PRMs, and use PRMs for test-time scaling and reinforcement learning. We summarize applications across math, code, text, multimodal reasoning, robotics, and agents, and review emerging benchmarks. Our goal is to clarify design spaces, reveal open challenges, and guide future research toward fine-grained, robust reasoning alignment.
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2510.08049 [cs.CL] |
| (or arXiv:2510.08049v3 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2510.08049 arXiv-issued DOI via DataCite |
Submission history
From: Congmin Zheng [view email]
[v1]
Thu, 9 Oct 2025 10:35:31 UTC (865 KB)
[v2]
Tue, 21 Oct 2025 14:21:25 UTC (865 KB)
[v3]
Wed, 29 Apr 2026 02:51:24 UTC (861 KB)