Double strategies for reproducing multimodal puns in interlinguistic translation: An experimental research


  • Oleksandr Rebrii* Vasyl Karazin National University of Kharkiv, Ukraine
  • Iryna Frolova Vasyl Karazin National University of Kharkiv, Ukraine
  • Inna Rebrii Ivan Kozhedub National Air Force University of Kharkiv, Ukraine



interlinguistic translation, intersemiotic translation, multimodal pun, retrospective experiment, translation strategy, verbal mode, visual mode


The research is dedicated to exposing the specifics of translating multimodal puns based on the retrospective experiment. A multimodal pun is defined as a creolized/ polycoded formation substantiated by a specific type of ambiguity consisting of two inhomogeneous semiotic modes. The material for our research was provided by verbal-visual puns functioning as separate texts. Each multimodal pun results from intersemiotic translation when the signs of one semiotic system are transformed into the signs of another. For the vast majority of multimodal puns, verbal signs are interpreted into pictures, but the opposite cannot be excluded either. The role of the visual component is twofold. It can be creative when the picture is part of the ambiguity mechanism, or it can be amplifying when the picture accentuates the verbal wordplay, not participating directly in the creation of ambiguity. Hence, our first hypothesis is that multimodal puns with an amplifying visual component are a lesser challenge for translators than those with a creative one due to the absence of the necessity to coordinate verbal and visual modes in the target text. The research aims to identify the strategies of interlinguistic translation of multimodal puns and the factors that determine them, particularly the impact of the visual mode on the translator’s decision-making. Hence, our second hypothesis is that multimodal puns require double strategies that would allow to correlate the reproduction of the elements of two different semiotic systems. The analysis of the subjects’ translations, as well as their reports received in the course of the delayed retrospective experiment, confirmed both hypotheses.

Disclosure Statement

The authors reported no potential conflicts of interest.

* Corresponding author: Oleksandr Rebrii,



Download data is not yet available.


Bernardini S. (2001). Think-aloud protocols in translation research. Achievements, limits, future prospects. Target, 13(2), 241–263.

Delabastita, D. (1996). Introduction. The Translator, 2(2), 127–139.

Dimitrova, B. E., Tiselius, E. (2014). Retrospection in Interpreting and Translation: Explaining the Process? MonTi Monografías de Traducción e Interpretación, Special Issue – Minding Translation, 177–200.

Eco, U. (1976). A Theory of Semiotics. Indiana University Press.

Ericsson K. A., Simon H. A. (1980). Verbal Reports as Data. Psychological Review, 87(3), 215–251.

Gile, D. (2004). Integrated Problem and Decision Reporting as a Translator Training Tool. The Journal of Specialised Translation, 2(2), 2–20. Retrieved from

Gottlieb, H. (2018). Semiotics and translation. In Kirsten Malmkjær (ed.). The Routledge Handbook of Translation Studies and Linguistics. (pp. 46–63). Routledge.

Hansen, G. (2005). Experience and Emotion in Empirical Translation Research with Think-Aloud and Retrospection. Meta, 50(2).

Huizinga, J. (1980). Homo Ludens. A Study of the Play-Element in Culture, Anscombe G. E. M (trans.). Routledge & Kegan Paul.

Jakobson, R. (1959). On Linguistic Aspects of Translation. In R. A. Brower (ed.). On Translation. (pp. 232–239). Harvard University Press.

Ketola, A. (2018). Word–Image Interaction in Technical Translation. Students Translating an Illustrated Text. Unpublished Academic Dissertation. University of Tampere.

Kourdis, E. (2020). Towards a Typology in Intersemiotic Translation from Verbal to Nonverbal and Polysemiotic Signs. In M. Kuleli, & D. Tuna (eds.). Conceptual Readings in Languages, Literature and Translation. (p. 79–99). Nobel.

Kristeva, J. (1984). Revolution in Poetic language, Waller M. (trans.). Columbia University Press.

Kussmaul, P., Tirkkonen-Condit, S. (1995). Think-Aloud Protocol Analysis in Translation Studies. TTR: traduction, terminologie, redaction, 8(1), 177–199. https://10.7202/037201ar

Leech, G. N. (1969). A Linguistic Guide to English Poetry. Longman.

Lemke J. (2002). Travels in hypermodality. Visual Communication 1(3), 299–325. https://10.1177/147035720200100303

Llado, R. (2002). La paraula revessa. Estudi sobre la traducciу dels jocs de mots. Servei de Publicacions de la Universitat Autonoma de Barcelona.

Marco, J. (2010). The Translation of wordplay in literary texts. Typology, techniques and factors in a corpus of English–Catalan source text and target text segments. Target, 22(2), 264–297.

O’Halloran, K. L. (2008). Systemic functional-multimodal discourse analysis (SF–MDA): constructing ideational meaning using language and visual imagery. Visual Communication, 7(4), 443–475.

Palumbo, J. (2009). Key Terms in Translation Studies. Continuum.

Pereira, N. M. (2008). Book Illustration as (Intersemiotic) Translation: Pictures Translating Words. Meta, 53(1), 104–119.

Rambæk, I. (2004). Translation strategies expressed in retrospective and concurrent verbal reports. Romansk Forum, 19, 7–19. Rebrii, O. (2012). Suchasni kontseptsii tvorchosti u perekladi [Modern conceptions of creativity in translation]. V.N. Karazin Kharkiv National University Publishers.

Rebrii, O., Rebrii, I., & Pieshkova, O. (2022). When words and images play together in a multimodal pun. Lublin Studies in Modern Languages and Literature, 46(2), 85–97.

Wittgenstein, L. (1986). Philosophical Investigations. Basil Blackwell.

Zavadska, O. V. (2016). Fenomen kreolizovanoho tekstu: aktualna problema suchasnykh linhvistychnykh doslidzhen [The Phenomenon of a Creolized Text: A Topical Issue of Modern Linguistic Research]. Linguistic Research, 43, 163–169.


Funny Science Jokes: Laughs for Scientists. Retrieved from:

Literaturoznavchyi Slovnyk–Dovidnyk [Dictionary–Directory of Literary Studies] (2007). Hromiak R. T., Kovaliv Yu. I. and Teremok V. I. (eds.). Akademia.

Solution Definition and Meaning. Merriam-Webster Online Dictionary and Thesaurus. Available from:

Tectonics Definition and Meaning. Merriam-Webster Online Dictionary and Thesaurus. Available from:

Zahnitko, A. (2012). Slovnyk suchasnoi linhvistyky: poniattia i terminy [The Dictionary of Modern Linguistics: Notions and Terms]. Donetsk National University.





Vol. 11

How to Cite

Rebrii, O., Frolova, I., & Rebrii, I. (2024). Double strategies for reproducing multimodal puns in interlinguistic translation: An experimental research. East European Journal of Psycholinguistics , 11(1).