Abstract:Information extraction from semi-structured business documents remains a critical challenge for enterprise management. This study evaluates the capability of general-purpose Large Language Models to extract structured information from Spanish electricity invoices without task-specific fine-tuning. Using a subset of the IDSEM dataset, we benchmark two architecturally distinct models, Gemini 1.5 Pro and Mistral-small, across 19 parameter configurations and 6 prompting strategies. Our experimental framework treats prompt engineering as the primary experimental variable, comparing zero-shot baselines against increasingly sophisticated few-shot approaches and iterative extraction strategies. Results demonstrate that prompt quality dominates over hyperparameter tuning: the F1-score variation across all parameter configurations is marginal, while the gap between zero-shot and the best few-shot strategy exceeds 19 percentage points. The best configuration (few-shot with cross-validation) achieves an F1-score of 97.61% for Gemini and 96.11% for Mistral-small, with document template structure emerging as the primary determinant of extraction difficulty. These findings establish that prompt design is the critical lever for maximizing extraction fidelity in LLM-based document processing, thereby providing an empirical framework for integrating general-purpose LLMs into business document automation.
| Comments: | 13 pages, 2 figures |
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2604.25927 [cs.CL] |
| (or arXiv:2604.25927v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2604.25927 arXiv-issued DOI via DataCite |
Submission history
From: Javier Sánchez [view email]
[v1]
Wed, 1 Apr 2026 12:53:13 UTC (454 KB)