Abstract:In this paper, we examine linguistic puzzles used in high school linguistics competitions, focusing on two common formats: Rosetta Stone and Match-Up. We propose a systematic procedure for converting existing Rosetta Stone puzzles into corresponding Match-Up counterparts. Because linguistic puzzle creation is complex and time-consuming, our method provides an efficient way to accelerate the generation of new puzzles. We evaluate the resulting Rosetta Stone-Match-Up pairs with both human participants and large language models (LLMs). Our results show that both expert human solvers and LLMs display an all-or-nothing pattern on Match-Up puzzles, either solving them completely or failing entirely. This work contributes a new dataset of paired puzzles and provides a detailed evaluation of puzzle difficulty across formats, offering insights into both human and machine linguistic reasoning.
| Comments: | Proceedings of the Fifteenth Language Resources and Evaluation Conference |
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2605.13408 [cs.CL] |
| (or arXiv:2605.13408v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2605.13408 arXiv-issued DOI via DataCite (pending registration) |
|
| Related DOI: | https://doi.org/10.63317/3yq3khhygtya
DOI(s) linking to related resources |
Submission history
From: Neh Majmudar [view email]
[v1]
Wed, 13 May 2026 12:03:35 UTC (92 KB)