Multimodal/Polycode Text as an Effectiveness Factor for Information Delivery on Social Media
DOI:
https://doi.org/10.17072/2073-6681-2025-3-66-79Keywords:
multimodal text; polycode text; multi-literacy; multimodal analysis; ER (engagement rate); modus; media topics.Abstract
The article is dedicated to the analysis of the effectiveness of polycode/multimodal texts in digital business communication. Particular attention is given to the role of the verbal component in multimodal messages. The study is novel in that it deals with new material (business communication), provides scientometric analysis, and evaluates the effectiveness (ER) of polycode/multimodal texts. The research material consists of 221 polycode/multimodal texts used in social media communications of small businesses. The methods utilized for material collection are continuous sampling and observation. The methods of analysis include scientometric analysis, content analysis, quantitative analysis, web analytics (ER, engagement rate), elements of discourse analysis and multimodal analysis. Scientometric analysis shows that Russian academic discourse mainly employs the terms creolized text and polycode text, which cannot be considered synonymous with the term multimodal. The study confirms the effectiveness of polycode/multimodal texts in correlation with the content format, modes, content type, and media topics. It has been established that the effectiveness of polycode/multimodal text is related not only to the variety of formats, types of content, and media topics but also to the conditions of the reader-viewer interaction with the text. It is necessary to establish correlations and patterns in the navigation within the virtual space of a text, in the creation and construction of text meanings in a sociocultural context, and in the design of reading methods and trajectories. The data obtained brings us back to the discussion about the importance of verbal and non-verbal codes in the transmission of information.References
Алексеев А. В. Инструментарий исследования протестных социально-политических кампаний (на примере движения Pray For Paris) // Вестник Пермского университета. Российская и зарубежная филология. 2025. Т. 17, вып. 1. С. 5–13. doi 10.17072/2073-6681-2025-1-5-13.
Белоедова А. В. Кожемякин Е. А., Тяжлов Я. И. Влияние вербального комментария на интерпретацию визуального медиасообщения: мультимодальный подход // Медиалингвистика. 2020. №7 (4). С. 445–461. doi 10.21638/spbu22.2020.406.
Добросклонская Т. Г. Медиалингвистика: системный подход к изучению языка СМИ. М.: Флинта: Наука, 2008. 263 с.
Ейгер Г. В., Юхт В. Л. К построению типологии текстов // Лингвистика текста: материалы науч. конф. при МГПИИЯ им. М. Тореза. М., 1974. Ч. 1. С. 103–109.
Загидуллина М. В. Современное состояние мультимодального анализа: к вопросу о перспективах метода // Научный результат. Социальные и гуманитарные исследования. 2023. Т. 9, № 1. С. 84–99. doi 10.18413/2408-932X-2023-9-1-0-7.
Интерпретация визуальной грамотности: Полное руководство по навыкам, 2024. URL: https://rolecatcher.com/ru/skills/hard-skills/information-skills/conducting-studies-investigations-and-examinations/interpret-visual-literacy/ (дата обращения: 25.05.2025).
Кресс Г. Социальная семиотика и вызовы мультимодальности // Политическая наука. 2016. № 3. С. 77–100.
Лужнова Н. В., Курбанов Ш. М. Типы контента и принципы создания контент-плана для продвижения коммерческого аккаунта в социальных сетях // Инновационная экономика: перспективы развития и совершенствования. 2023. № 8(74). С. 104–109.
Назметдинова И. С., Куличев К. М., Столярова С. С. Понятия «медиаграмотность» и «медиаобразование» в отечественной и зарубежной науке // Проблемы современного педагогического образования: сб. науч. тр. Ялта: РИО ГПА, 2024. Вып. 85, ч. 3. С. 236–239.
Новоспасская Н. В., Дугалич Н. М. Терминосистема теории поликодовых текстов // Русистика. 2022. Т. 20, № 3. С. 298–311. doi 10.22363/2618-8163-2022-20-3-298-311.
Сорокин Ю. А., Тарасов Е. Ф. Креолизованные тексты и их коммуникативная функция // Оптимизация речевого воздействия. М.: Высш. шк., 1990. С. 180–186.
Чабан М. А. Теоретическое обоснование термина «визуальная грамотность» в условиях цифровизации общества // Известия Саратовского университета. Новая серия. Серия: Философия. Психология. Педагогика. 2022. Т. 22, вып. 2. С. 218–222. doi 10.18500/1819-7671-2022-22-2-218-222.
Álvarez J. Meaning Making and Communication in the Multi-modal Age: Ideas for Language Teachers // Colomb. Appl. Linguist. J. 2016. № 18(1). P. 98–115. doi 10.14483/calj.v18n1.8403.
Bazalgette C., Buckingham D. Literacy, media and multimodality: a critical response // Literacy. 2013. Vol. 47. № 2. P. 95–102.
Bul I. V., Antonova K. N. On the issue of classification of multimodal texts of a new order // Материалы V Всероссийской научно-практической конференции с международным участием «Диалог поколений: Изучаем. Обучаем. Учимся» (23-24 апреля 2024 г.): в 2 ч. / под общ. ред. В. В. Кирилловой. СПб.: ВШТЭ СПбГУПТД, 2024. Ч. 2. С. 59–62.
Freebody P., Luke A. Literacies programs: Debates and demands in cultural context // Prospect: Australian Journal of TESOL, 1990. № 5(7). P. 7–16.
Introduction to Multimodal Analysis. Second Edition by Per Ledin and David Machin. Bloomsbury Academic, 2020. 256 р.
Kress G. Multimodal discourse analysis // The Routledge Handbook of Discourse Analysis Routledge. 2012. P. 35–50. doi 10.4324/9780203809068.
Machin D. Introduction to Multimodal Analysis. London: Bloomsbury Academic, 2007. 224 p.
McKay S. Media and language: Overview // Encyclopedia of language and linguistics / ed. by Yu. I. Alexandrov et al. 2nd ed. Oxford: Elsevier, 2006. Vol. 8. P. 597–602.
Metros S. E. The educator's role in preparing visually literate learners // Theory into Practice. 2008. Vol. 47. P. 102–109.
Mitchell W. J. T. Visual literacy or literary visualcy // Visual literacy / ed. by J. Elkins. New York: Routledge, 2008. Vol. 39. № 1. P. 11–14.
Pernice K. Banner Blindness Revisited: Users Dodge Ads on Mobile and Desktop. 2018. URL: https://www.nngroup.com/articles/banner-blindness-old-and-new-findings/ (дата обращения: 25.05.2025).
Serafini F. Expanding the four resources model: reading visual and multi-modal texts // Pedagogies. An International Journal. 2012. Vol. 7. № 2. P. 150–164. doi 10.1080/1554480X.2012.656347
The New London Group. A Pedagogy of Multiliteracies: Designing Social Futures // Harvard Educational Review. 1996. Vol. 66. № 1. P. 60–92.
Trimbur J. Multiliteracies: Literacy Learning and the Design of Social Futures: Review // College Composition and Communication. 2001. Vol. 52. № 4. Р. 659-662. doi 10.2307/358703.
World’s first global standard for digital-literacy and skills launched by the coalition for digital intelligence. 2019 // DQ Institute. URL: https://www.dqinstitute.org/news-post/worlds-first-global-standard-for-digital-literacy-and-skills-launched-by-the-coalition-for-digital-intelligence/ (дата обращения: 25.05.2025).
References
Alekseev A. V. Instrumentariy issledovaniya protestnykh sotsial’no-politicheskikh kampaniy (na primere dvizheniya Pray for Paris) [Tools for studying protest socio-political campaigns (a case study of the Pray for Paris Movement)]. Vestnik Permskogo universiteta. Rossiyskaya i zarubezhnaya filologiya [Perm University Herald. Russian and Foreign Philology], 2025, vol. 17, issue 1, pp. 5-13. doi 10.17072/2073-6681-2025-1-5-13. (In Russ.)
Beloedova A. V., Kozhemyakin E. A., Tyazhlov Y. I. Vliyanie verbal'nogo kommentariya na interpretatsiyu vizual'nogo mediasoobshcheniya: mul'timodal'nyy podkhod [Impact of verbal comment on interpretation of visual message: Multimodal approach]. Media Linguistics, 2020, vol. 7 (4), рр. 445-461. doi 10.21638/spbu22.2020.406. (In Russ.)
Dobrosklonskaia T. G. Medialingvistika: sistemnyy podkhod k izucheniyu yazyka SMI [Medialinguistics: A Systemic Approach to the Study of the Language of the Media.] Moscow, Flinta: Nauka Publ., 2008. 263 p. (In Russ.)
Yeiger G. V., Yukht V. L. K postroeniyu tipologii tekstov [On the construction of a typology of texts]. Lingvistika teksta: materialy nauch. konf. pri MGPIIYa im. M. Toreza [Text Linguistics: Proceedings of the Scientific Conference at the Maurice Thorez Moscow State Pedagogical Institute of Foreign Languages]. Moscow, 1974, pt. 1, рp. 103-109. (In Russ.)
Zagidullina M. V. Sovremennoe sostoyanie mul'timodal'nogo analiza: k voprosu o perspektivakh metoda [The current state of multimodal analysis: on the question of the prospects of the method]. Nauchnyy rezul'tat. Sotsial'nye i gumanitarnye issledovaniya [Research Result. Social Studies and Humanities], 2023, vol. 9, issue 1, рр. 84-99. doi 10.18413/2408-932X-2023-9-1-0-7. (In Russ.)
Interpretatsiya vizual'noy gramotnosti: Polnoe rukovodstvo po navykam [Interpreting visual literacy: A complete guide to skills], 2024. Available at: https://rolecatcher.com/ru/skills/hard-skills/information-skills/conducting-studies-investigations-and-examinations/interpret-visual-literacy/ (accessed 25 May 2025). (In Russ.)
Kress G. Sotsial’naya semiotika i vyzovy mul’timodal’nosti [Social semiotic and the challenge of multimodality]. Politicheskaya nauka [Political Science], 2016, issue 3, pp. 77-100. (In Russ.)
Luzhnova N. V., Kurbanov S. M. Tipy kontenta i printsipy sozdaniya kontent-plana dlya prodvizheniya kommercheskogo akkaunta v sotsial'nykh setyakh [Types of content and principles of creating a content plan for promoting a commercial account on social media]. Innovatsionnaya ekonomika: perspektivy razvitiya i sovershenstvovaniya [Innovative Economy: Prospects for the Development and Improvement], 2023, vol. 8 (74), pp. 104-109. (In Russ.)
Nazmetdinova I. S., Kulichev K. M., Stolyarova S. S. Ponyatiya ʽmediagramotnostˮ i ʽmediaobrazovanieʼ v otechestvennoy i zarubezhnoy nauke [The concepts of ʽmedia literacyʼ and ʽmedia educationʼ in domestic and foreign science]. Problemy sovremennogo pedagogicheskogo obrazovaniya: sbornik nauchnykh trudov [Issues of Modern Pedagogical Education: Collection of Scientific Works]. Yalta, RIO GPA Publ., 2024, issue 85, pt. 3, pp. 236-239. (In Russ.)
Novospasskaya N. V., Dugalich N. M. Terminosistema teorii polikodovykh tekstov [Terminological system of the polycode text theory]. Rusistika [Russian Language Studies], 2022, vol. 20, issue 3, рр. 298-311. doi 10.22363/2618-8163-2022-20-3-298-311. (In Russ.)
Sorokin Yu. A., Tarasov E. F. Kreolizovannye teksty i ikh kommunikativnaya funktsiya [Creolized texts and their communicative function]. Optimizatsiya rechevogo vozdeystviya [The Optimization of Speech Impact]. Moscow, Vysshaya shkola Publ., 1990, pp. 180-186. (In Russ.)
Chaban M. A. Teoreticheskoe obosnovanie termina ʽvizual'naya gramotnostˮ v usloviyakh tsifrovizatsii obshchestva [Theoretical substantiation of the term ʽvisual literacyʼ in the context of the digitalization of society]. Izvestiya Saratovskogo universiteta. Novaya seriya. Seriya: Filosofiya. Psikhologiya. Pedagogika [Izvestiya of Saratov University. Philosophy. Psychology. Pedagogy], 2022, vol. 22, issue 2, pp. 218-222. doi 10.18500/1819-7671-2022-22-2-218-222. (In Russ.)
Álvarez J. Meaning making and communication in the multi-modal age: Ideas for language teachers. Colombian Applied Linguistics Journal, 2016, issue 18 (1), pp. 98-115. doi 10.14483/calj.v18n1.8403. (In Eng.)
Bazalgette C., Buckingham D. Literacy, media and multimodality: A critical response. Literacy, 2013, vol. 47, issue 2, pp. 95-102. (In Eng.)
Bul I. V., Antonova K. N. On the issue of classification of multimodal texts of a new order. Materialy V Vserossiyskoy nauchno-prakticheskoy konferentsii s mezhdunarodnym uchastiem ʽDialog pokoleniy: Izuchaem. Obuchaem. Uchimsyaʼ (23-24 aprelya 2024 g.) [Proceedings of the V All-Russian Research and Practice Conference with International Participation ʽDialogue of Generations: Learn. Teach. Studyʼ (April 20-21, 2024)]: in 2 pts. Ed. by V. V. Kirillova. St. Petersburg, HSTE SPbSUITD Press, 2024, pt. 2, pp. 59-62. (In Eng.)
Freebody P., Luke A. Literacies programs: Debates and demands in cultural context. Prospect: Australian Journal of TESOL, 1990, issue 5 (7), pp. 7-16. (In Eng.)
Introduction to Multimodal Analysis. 2nd ed. by Per Ledin and David Machin. Bloomsbury Academic, 2020. 256 p. (In Eng.)
Kress G. Multimodal discourse analysis. The Routledge Handbook of Discourse Analysis Routledge, 2011, pp. 35-50. doi 10.4324/9780203809068 (accessed 25 May 2025). (In Eng.)
Machin D. Introduction to Multimodal Analysis. London, Bloomsbury Academic, 2007. 224 p. (In Eng.)
McKay S. Media and language: Overview. Encyclopedia of Language and Linguistics. Ed. by Yu. I. Alexandrov et al. 2nd ed. Oxford, Elsevier, 2006, vol. 8, pp. 597-602. (In Eng.)
Metros S. E. The educator's role in preparing visually literate learners. Theory into Practice, 2008, vol. 47, pp. 102-109. (In Eng.)
Mitchell W. J. T. Visual literacy or literary visualcy. Ed. by J. Elkins. Visual Literacy. New York, Routledge, 2008, vol. 39, issue 1, pp. 11-14. (In Eng.)
Pernice K. Banner blindness revisited: Users dodge ads on mobile and desktop, 2018. Available at: https://www.nngroup.com/articles/banner-blindness-old-and-new-findings/ (accessed 25 May 2025). (In Eng.)
Serafini F. Expanding the four resources model: reading visual and multi-modal texts. Pedagogies. An International Journal, 2012, vol. 7, issue 2, pp. 150-164. doi 10.1080/1554480X.2012.656347. (In Eng.)
The New London Group. A pedagogy of multiliteracies: Designing social futures. Harvard Educational Review, 1996, vol. 66, issue 1, рр. 60-92. (In Eng.)
Trimbur J. Multiliteracies: Literacy Learning and the Design of Social Futures. College Composition and Communication, 2001, vol. 52, no 4, pp. 659-662. doi 10.2307/358703. (In Eng.)
Worldʼs first global standard for digital-literacy and skills launched by the coalition for digital intelligence, 2019. DQ Institute. Available at: https://www.dqinstitute.org/news-post/worlds-first-global-standard-for-digital-literacy-and-skills-launched-by-the-coalition-for-digital-intelligence/ (accessed 25 May 2025). (In Eng.)
