Abstract:Malaysian English is a low resource creole language, where it carries the elements of Malay, Chinese, and Tamil languages, in addition to Standard English. Named Entity Recognition (NER) models underperform when capturing entities from Malaysian English text due to its distinctive morphosyntactic adaptations, semantic features and code-switching (mixing English and Malay). Considering these gaps, we introduce MENmBERT and MENBERT, a pre-trained language model with contextual understanding, specifically tailored for Malaysian English. We have fine-tuned MENmBERT and MENBERT using manually annotated entities and relations from the Malaysian English News Article (MEN) Dataset. This fine-tuning process allows the PLM to learn representations that capture the nuances of Malaysian English relevant for NER and RE tasks. MENmBERT achieved a 1.52\% and 26.27\% improvement on NER and RE tasks respectively compared to the bert-base-multilingual-cased model. Although the overall performance of NER does not have a significant improvement, our further analysis shows that there is a significant improvement when evaluated by the 12 entity labels. These findings suggest that pre-training language models on language-specific and geographically-focused corpora can be a promising approach for improving NER performance in low-resource settings. The dataset and code published in this paper provide valuable resources for NLP research work focusing on Malaysian English.
| Comments: | Accepted in 9th Workshop on Representation Learning for NLP (Rep4NLP) at ACL 2024 |
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2407.01374 [cs.CL] |
| (or arXiv:2407.01374v2 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2407.01374 arXiv-issued DOI via DataCite |
Submission history
From: Mohan Raj Chanthran Mr [view email]
[v1]
Mon, 1 Jul 2024 15:26:03 UTC (8,126 KB)
[v2]
Sat, 30 May 2026 10:02:36 UTC (8,126 KB)