Abstract:Constructing scientific multimodal document reasoning datasets for foundation model training involves an inherent trade-off among scale, faithfulness, and realism. To address this challenge, we introduce the synthesize-and-reground framework, a two-stage pipeline comprising: (1) Claim-Centric QA Synthesis, which generates faithful, isolated QA pairs and reasoning on focused segments, and (2) Document-Scale Regrounding, which programmatically re-embeds these pairs into full-document tasks to ensure realistic complexity. Using this framework, we construct SciMDR, a large-scale training dataset for cross-modal comprehension, comprising 300K QA pairs with explicit reasoning chains across 20K scientific papers. We further construct SciMDR-Eval, an expert-annotated benchmark to evaluate multimodal comprehension within full-length scientific workflows. Experiments demonstrate that models fine-tuned on SciMDR achieve significant improvements across multiple scientific QA benchmarks, particularly in those tasks requiring complex document-level reasoning.
| Comments: | ACL 2026 |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV) |
| Cite as: | arXiv:2603.12249 [cs.CL] |
| (or arXiv:2603.12249v2 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2603.12249 arXiv-issued DOI via DataCite |
Submission history
From: Ziyu Chen [view email]
[v1]
Thu, 12 Mar 2026 17:57:52 UTC (4,834 KB)
[v2]
Wed, 29 Apr 2026 04:59:09 UTC (5,136 KB)