The multimodal approach to the translation of exhibition texts – a pedagogical example

Abstract

Defined as “the use of several semiotic modes in the design of a semiotic product or event” , “multimodality” enables a comprehensive interpretation of the semiotic complexity of texts. The multimodal perspective enables translation studies, traditionally emphasised the verbal, to reconsider the re-semiotic aspects of meaning-making. By emphasising the multimodal semiotic mode of communication, this study explores the role of multimodality in the translations of exhibition texts by examining the impact of non-verbal and semiotic factors, particularly visual stimuli, in a pedagogical setting. As a textual domain embedded in the museum as a multimodal site, exhibition texts are accompanied by abundant visual and audial elements and interactive dimensions. This allows for a considerable degree of latitude in exploring the implication of meaning-making, particularly resemiotisation, in translation under the impact of multimodal interaction. Through qualitative textual comparison, this paper analyses the selected examples of students’ translations of the exhibition texts that were assigned; this research seeks to investigate 1) how the non-verbal or semiotic elements are translated from one into another; 2) how the consideration of multimodal resources can complement or enhance the initially obscure or ambiguous source textual meaning in the translation, and hence add interpretive complexity to the translation. The analysis focuses on cases that demonstrate a prominent integration of non-verbal resources, including those from painting, interactive video, and audio. We also refer to systemic-functional linguistics, offering a framework for the investigation into semiosis involving visual stimuli and language, to help identify to what extent the visual stimuli can shape the translation solutions by influencing the specific words or structures used in the final product. This research aims not only to accentuate the idea that the translation is more than mere verbal transference, but also to shed light on the functional relationship between the semiotic resources and their medial realisation.

Keywords

Multimodality, re-semiotisation, translation of exhibition texts