Abstract:This work investigates whether knowledge-driven large language model (LLM)-based storytelling can support purposeful narrative interaction with a digital companion for older adults. To address known limitations of LLMs, including hallucinations and limited transparency, we present a reflective storytelling agent integrating knowledge graphs, user modelling, argumentation theory, and argument mining to guide and inspect narrative generation.
The study consisted of two phases. Phase I employed participatory design involving 11 domain experts in a formative evaluation that informed iterative refinement. The resulting system generates narratives grounded in structured user models representing health-promoting activities and motivations. Phase II involved 55 older adults evaluating persona-based narratives across four prompts and two creativity levels. Participants assessed perceived purpose, usefulness, cultural relatability, and inconsistencies. The system additionally computed hallucination-risk indicators to evaluate generated narratives.
Participants recognised personally relevant purposes in roughly two thirds of narratives, while argument-based purposes were identified in around half of these cases. Cultural recognisability strongly influenced willingness to use the functionality, whereas minor inconsistencies were often tolerated when narratives remained understandable and personally relevant. Narratives with higher hallucination-risk indicators were more often perceived as inconsistent, while higher argument-quality indicators tended to co-occur with higher clarity and meaningfulness ratings.
Overall, the study positions argument mining as a reflective inspection mechanism for comparing formal grounding signals with human evaluations in health-oriented LLM storytelling for older adults.
| Comments: | Submitted to ACM Transactions on Intelligent Systems and Technology (TIST) |
| Subjects: | Artificial Intelligence (cs.AI) |
| ACM classes: | H.5.2; I.2.7 |
| Cite as: | arXiv:2605.10531 [cs.AI] |
| (or arXiv:2605.10531v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2605.10531 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Jayalakshmi Baskar [view email]
[v1]
Mon, 11 May 2026 13:17:31 UTC (669 KB)