Contextual Polysemy Resolution in Machine Translation: Evaluating LLMs on the Peruvian Variety of Spanish
DOI:
https://doi.org/10.17072/2073-6681-2026-3-56-64Keywords:
machine translation; large language models; polysemy; Peruvian variety of Spanish; culture-specific vocabulary; translation quality; context.Abstract
. The article addresses a topical issue in machine translation of culture-specific vocabulary from national variants of Spanish using large language models (LLMs). The main focus is on the ability of LLMs to resolve polysemy in context when translating lexical units of the Peruvian variant of Spanish into Russian. The research problem stems from insufficient study and understanding of how cultural bias in LLMs affects translation quality of polysemous vocabulary, as well as from the gap between lexicographically recorded and unrecorded meanings in model training data. The study aims to evaluate the capability of three LLMs (Gemini 2.5 Pro, Manus, Claude Sonnet 4.5) to translate polysemous lexemes in context and identify error patterns. An adapted four-parameter methodology by Park and Padó was applied, introducing the Cultural Adequacy parameter to assess preservation of cultural components of meaning. The material comprised three polysemous lexemes in six authentic contexts extracted from digital sources such as blogs and social media. Results demonstrate that models less prone to hallucinations achieve 24 % higher translation quality. A significant gap was identified in handling academically unrecorded meanings: Claude Sonnet 4.5 shows a 28 % quality decline, whereas Gemini 2.5 Pro and Manus maintain relative stability. Based on the findings, practical recommendations were developed for selecting LLMs for machine translation tasks involving culturally marked vocabulary of the Peruvian national variety of Spanish.
References
Diccionario de americanismos. Asociación de Academias de la Lengua Española. URL: https://www.asale.org/damer/cojinova (дата обращения: 06.11.2025).
Diccionario de americanismos. Asociación de Academias de la Lengua Española. URL: https://www.asale.org/damer/¡chaufa! (дата обра-щения: 06.11.2025).
Diccionario de la lengua española. Real Academia Española. URL: https://dle.rae.es/mote (дата обращения: 06.11.2025).
Diccionario de la lengua española. Real Academia Española. URL: https://dle.rae.es/peruanismo (дата обращения: 27.09.2025).
Diccionario Libre. URL: https://diccionariolibre.com/definicion/cojinova (дата обращения: 06.11.2025).
Список литературы
Контрерас Сарриа Х. О. Перуанский вариант испанского языка: фонетические, грамматические, лексические особенности // Вестник Челябинского государственного университета. Филологические науки. 2022. № 3 (461), вып. 128. С. 43–50. doi 10.47475/1994-2796-2022-10306
Межова М. В. Проблема культурологической эквивалентности слова в переводе художественного текста // Вестник Кемеровского государственного университета культуры и искусств. 2012. № 20. С. 78–82.
Селимов А. А. Оценка лингвокультурной компетенции больших языковых моделей на материале перуанского варианта испанского языка // Профессионально ориентированный перевод: реальность и перспективы: сб. науч. тр. (Москва, 14–15 апреля 2025 г.). Чебоксары: Среда, 2025. С. 115–126.
Селимов А. А., Федорова И. К. Понимание и перевод лексем различных национальных вариантов испанского языка // Индустрия перевода: материалы XVI Междунар. науч. конф. (Пермь, 25–26 апреля 2024 г.). Пермь: Перм. нац. исслед. политехн. ун-т, 2025. С. 93–99.
Тарнаева Л. П. О передаче культурно-специфических смыслов при переводе: лингво-методический аспект проблемы // Альманах современной науки и образования. 2008. № 2-3. С. 217–218.
Aksoy M. Whose morality do they speak? Unraveling cultural bias in multilingual language models // Natural Language Processing Journal. 2025. Vol. 12. Article 100172. doi 10.1016/j.nlp.2025.100172
Harper K. E. Minority Dialects in Contact: Peruvian Spanish in the United States: PhD thesis in Hispanic Studies. College Station: Texas A&M University, 2019. 276 p.
Huang L. et al. A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions / L. Huang, W. Yu, W. Ma, W. Zhong, Z. Feng, H. Wang, Q. Chen, W. Peng, X. Feng, B. Qin, T. Liu // ACM Transactions on Information Systems. 2025. Vol. 43, № 2. Article 42. doi 10.1145/3703155
Lommel A., Uszkoreit H., Burchardt A. Multi-dimensional Quality Metrics (MQM): A Frame-work for Declaring and Describing Translation Quality Metrics // Tradumàtica: tecnologies de la traducció. 2014. № 12. P. 455–463. doi 10.5565/rev/tradumatica.77
Naveen P., Trojovský P. Overview and chal-lenges of machine translation for contextually appropriate translations // iScience. 2024. Vol. 27, № 10. Article 110878. doi 10.1016/j.isci.2024.110878
Nord C. Translating as a Purposeful Activity: Functionalist Approaches Explained. 2nd ed. London: Routledge, 2018. 184 p. doi 10.4324/9781351189354
Pan E. et al. Analyzing Dialectical Biases in LLMs for Knowledge and Reasoning Benchmarks / E. Pan, A. S. G. Choi, M. ter Hoeve, S. Seto, A. Koenecke // Findings of the Association for Computational Linguistics: EMNLP 2025. Suzhou: Association for Computational Linguistics, 2025. P. 20882–20893. doi 10.18653/v1/2025.findings-emnlp.1139
Park D., Padó S. Multi-Dimensional Machine Translation Evaluation: Model Evaluation and Resource for Korean // Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024). Torino: ELRA and ICCL, 2024. P. 11723–11744.
Portillo López F. Imperfect Language, Artificial Intelligence, and the Human Mind: An Interdisciplinary Approach to Linguistic Errors in Native Spanish Speakers // arXiv preprint. 2025. URL: https://arxiv.org/abs/2511.01615 (дата обращения: 20.11.2025). doi 10.48550/arXiv.2511.01615
Waddington C. Should translations be assessed holistically or through error analysis? // HERMES - Journal of Language and Communication in Business. 2001. Vol. 14, № 26. P. 15–37. doi 10.7146/hjlcb.v14i26.25637
Yao B. et al. Benchmarking Machine Transla-tion with Cultural Awareness / B. Yao, M. Jiang, T. Bobinac, D. Yang, J. Hu // Findings of the Association for Computational Linguistics: EMNLP 2024. Miami: Association for Computational Linguistics, 2024. P. 13078–13096. doi 10.18653/v1/2024.findings-emnlp.765
References
Contreras Sarria J. O. Peruvian Spanish: Phonetic, grammatical, lexical features. Bulletin of Chelyabinsk State University. Philological Sciences, 2022, vol. 3 (461), issue 128, pp. 43–50. doi 10.47475/1994-2796-2022-10306 (In Russ.)
Mezhova M. V. The problem of the cultural equivalence of a word in the translation of a literary text. Bulletin of Kemerovo State University of Culture and Arts, 2012, issue 20, pp. 7882. (In Russ.)
Selimov A. A. Assessment of linguocultural competence of large language models based on the Peruvian variant of Spanish. Professionally oriented translation: Reality and prospects: a collection of scientific papers (Moscow, April 14-15, 2025). Cheboksary, Sreda Publ., 2025, pp. 115126. (In Russ.)
Selimov A. A., Fedorova I. K. Understanding and translating lexemes of the national varieties of the Spanish language. Translation industry: Proceedings of XVI International Scientific Conference (Perm, April 25-26, 2024). Perm, Perm National Research Polytechnic University Press, 2025, pp. 9399. (In Russ.)
Tarnaeva L. P. On the translation of culturally specific meanings: linguo-methodological aspect of the problem. Almanac of Modern Science and Education, 2008, issue 2-3, pp. 217218. (In Russ.)
Aksoy M. Whose morality do they speak? Unraveling cultural bias in multilingual language models. Natural Language Processing Journal, 2025, vol. 12, article 100172. doi 10.1016/j.nlp.2025.100172 (In Eng.)
Harper K. E. Minority dialects in contact: Peruvian Spanish in the United States. PhD thesis in Hispanic Studies. College Station, Texas A&M University, 2019. 276 p. (In Eng.)
Huang L., et al. A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions. ACM Transactions on Information Systems, 2025, vol. 43, issue 2, article 42. doi 10.1145/3703155 (In Eng.)
Lommel A., Uszkoreit H., Burchardt A. Multi-dimensional quality metrics (MQM): A framework for declaring and describing translation quality metrics. Tradumàtica: Tecnologies de la traducció, 2014, issue 12, pp. 455463. doi 10.5565/rev/tradumatica.77 (In Eng.)
Naveen P., Trojovský P. Overview and chal-lenges of machine translation for contextually appropriate translations. iScience, 2024, vol. 27, issue 10, article 110878. doi 10.1016/j.isci.2024.110878 (In Eng.)
Nord C. Translating as a purposeful activity: Functionalist approaches explained. 2nd ed. London, Routledge, 2018. 184 p. doi 10.4324/9781351189354 (In Eng.)
Pan E. et al. Analyzing dialectical biases in LLMs for knowledge and reasoning benchmarks. Findings of the Association for Computational Linguistics: EMNLP 2025. Suzhou, Association for Computational Linguistics, 2025, pp. 2088220893. doi 10.18653/v1/2025.findings-emnlp.1139 (In Eng.)
Park D., Padó S. Multi-dimensional machine translation evaluation: Model evaluation and re-source for Korean. Proceedings of the 2024 Joint International Conference on Computational Lin-guistics, Language Resources and Evaluation (LREC-COLING 2024). Torino, ELRA and ICCL, 2024, pp. 1172311744. (In Eng.)
Portillo López F. Imperfect language, artificial intelligence, and the human mind: An interdisciplinary approach to linguistic errors in native Spanish speakers. arXiv preprint, 2025. URL: https://arxiv.org/abs/2511.01615 (accessed 20 Nov 2025). doi 10.48550/arXiv.2511.01615 (In Eng.)
Waddington C. Should translations be assessed holistically or through error analysis? HERMES - Journal of Language and Communication in Business, 2001, vol. 14, issue 26, pp. 1537. doi 10.7146/hjlcb.v14i26.25637 (In Eng.)
Yao B., et al. Benchmarking machine translation with cultural awareness. Findings of the Association for Computational Linguistics: EMNLP 2024. Miami, Association for Computational Linguistics, 2024, pp. 1307813096. doi 10.18653/v1/2024.findings-emnlp.765 (In Eng.)
