Abstract:Autonomous LLM agents increasingly operate in long-horizon, interactive settings where success depends on reusing experience accumulated over extended histories. However, existing agent memory systems are fundamentally constrained by text-context budgets: storing or revisiting raw trajectories is prohibitively token-expensive, while summarization and text-only retrieval trade token savings for information loss and fragmented evidence. To address this limitation, we propose Optical Context Retrieval Memory (OCR-Memory), a memory framework that leverages the visual modality as a high-density representation of agent experience, enabling retention of arbitrarily long histories with minimal prompt overhead at retrieval time. Specifically, OCR-Memory renders historical trajectories into images annotated with unique visual identifiers. OCR-Memory retrieves stored experience via a \emph{locate-and-transcribe} paradigm that selects relevant regions through visual anchors and retrieves the corresponding verbatim text, avoiding free-form generation and reducing hallucination. Experiments on long-horizon agent benchmarks show consistent gains under strict context limits, demonstrating that optical encoding increases effective memory capacity while preserving faithful evidence recovery.
| Comments: | Accepted to ACL 2026 (Main Conference) |
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2604.26622 [cs.CL] |
| (or arXiv:2604.26622v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2604.26622 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Jinze Li [view email]
[v1]
Wed, 29 Apr 2026 12:49:30 UTC (1,300 KB)