Abstract:Large Language Models (LLMs) are increasingly employed in enterprise question-answering (QA) systems, requiring adaptation to domain-specific knowledge. Among the most prevalent methods for incorporating such knowledge are Retrieval-Augmented Generation (RAG) and fine-tuning (FT). Yet, from a cost-accuracy trade-off perspective, it remains unclear which approach best suits industry scenarios. This study examines the impact of RAG and FT on two closed datasets specific to the automotive industry, assessing answer quality and operational costs. We extend the Cost-of-Pass framework proposed by Erol et al. (arXiv:2504.13359) to jointly assess output quality, generation cost, and user interaction cost. Our findings reveal that while premium models perform best out of the box, open-source models can achieve comparable quality when enhanced with RAG. Overall, RAG emerges as the most effective and cost-efficient adaptation method for both closed- and open-source models.
| Comments: | Accepted at AAAI 2026 Workshop on New Frontiers in Information Retrieval |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2605.09533 [cs.CL] |
| (or arXiv:2605.09533v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2605.09533 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Jakob Sturm [view email]
[v1]
Sun, 10 May 2026 13:35:16 UTC (107 KB)