Show Less
Restricted access

Building Bridges for Multimodal Research

International Perspectives on Theories and Practices of Multimodal Analysis


Edited By Janina Wildfeuer

While multimodality is one of the most influential semiotic theories for analysing media artefacts, the concepts of this theory are heterogeneous and widespread. The book takes the differences between approaches in Germany and those in international contexts as a starting point, offering new insights into the analysis of multimodal documents. It features contributions by researchers from more than 15 nations and various disciplines, including theoretical reflections on multimodality, thoughts about methodological, empirical, and experimental approaches as well as analyses of various multimodal artefacts.
Show Summary Details
Restricted access

Spotlight on the Image: Adding the Missing Visual Piece to the Multimodal Puzzle


1 Introduction

The visual basis of multimodality’s production, reception, and analytical processes is uncontested throughout the literature on the subject. Multimodal documents are designed, created, disseminated, perceived, and analysed with the unavoidable reliance on human vision and with the assumptions of varying degrees of pre-existing visual expertise to aid each of these processes. Most of the popular analytical approaches in the field, however, are quite far removed from the realms of visual communication and visual studies. This chapter’s goal is to bring the visual up front and centre in multimodal document analysis and map out the transdisciplinary synergies which can support visually based content analysis.

The text reviews the disciplinary cornerstones, upon which this new analytical architecture rests, starting with the methodological scarcity typically found in visual studies proper. For all its lacks, image science nevertheless has a unique advantage: its inherent multimodal nature, which allows it to tackle complex content-analytical questions naturally. Therefore, the chapter addresses the following questions: How can we combine the analytical depth of visual content analysis with the strict annotation principles of corpus linguistics into a new, visually based multimodal annotation model? What are the theoretical and methodological synergies behind combining these two main approaches to document analysis? In the context of a specific research project, why is such a visually based approach to multimodal analysis superior to already available annotation tools?

To address the questions above, the chapter presents an augmentation of methods and principles with...

You are not authenticated to view the full text of this chapter or article.

This site requires a subscription or purchase to access the full text of books or journals.

Do you have any questions? Contact us.

Or login to access all content.