Abstract:Deep search has recently emerged as a promising paradigm for enabling agents to retrieve fine-grained historical information without heavy memory pre-managed. However, existing deep search agents for memory system repeat past error behaviors because they fail to learn from the prior high- and low-quality search trajectories. To address this limitation, we propose R^2-Mem, a reflective experience framework for memory search systems. In the offline stage, a Rubric-guided Evaluator scores low- and high-quality steps in historical trajectories, and a self-Reflection Learner distills the corresponding abstract experience. During the online inference, the retrieved experience will guide future search actions to avoid repeated mistakes and maintain high-quality behaviors. Extensive experiments demonstrate that R^2-Mem consistently improves both effectiveness and efficiency over strong baselines, improving F1 scores by up to 22.6%, while reducing token consumption by 12.9% and search iterations by 20.2%. These results verify that R^2-Mem provides a RL-free and low-cost solution for self-improving LLM agents.
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2605.13486 [cs.CL] |
| (or arXiv:2605.13486v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2605.13486 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Xinyuan Wang [view email]
[v1]
Wed, 13 May 2026 13:09:36 UTC (810 KB)