Show Less
Restricted access

Building Bridges for Multimodal Research

International Perspectives on Theories and Practices of Multimodal Analysis

Series:

Edited By Janina Wildfeuer

While multimodality is one of the most influential semiotic theories for analysing media artefacts, the concepts of this theory are heterogeneous and widespread. The book takes the differences between approaches in Germany and those in international contexts as a starting point, offering new insights into the analysis of multimodal documents. It features contributions by researchers from more than 15 nations and various disciplines, including theoretical reflections on multimodality, thoughts about methodological, empirical, and experimental approaches as well as analyses of various multimodal artefacts.
Show Summary Details
Restricted access

Cognitively Oriented Semiotics as a Common Descriptive Framework for Pictorial and Verbal Representation

Extract

| 77 →

Alina Kwiatkowska

Cognitively-Oriented Semiotics as a Common Descriptive Framework for Pictorial and Verbal Representations1

1 A new descriptive framework for multimodal messages

As the art theorist W. J. T. Mitchell has stated perceptively, today “all media are mixed media, all arts composite arts” (Mitchell 1994: 107). There is increasing awareness of the multimodal nature of communication. However, as has been pointed out in the introduction to this volume, theoretical reflection on multimodality still seems to be in its early stages, with varying perspectives on the terminology and methodology of the field. Encouraged by the editor’s invitation to ponder on the ways of meeting the challenge posed by multimodal messages, I would like to contribute to the discussion by calling attention to the theoretical framework that could be of great help in their description and analysis.

I would like to argue that the ideal general framework facilitating research on multimodality should have a cognitivist orientation. Due to my background in linguistics, my point of departure will be the version of cognitivism worked out by researchers on language, notably Ronald Langacker, Leonard Talmy, George Lakoff, and Mark Turner. Admittedly, with the exception of Turner, who has marginally touched upon images, and some researchers on visual figures, who will be mentioned further on in this paper, those authors have up to now restricted their attention to the verbal mode. This is perhaps why cognitivism has been largely overlooked in the context of...

You are not authenticated to view the full text of this chapter or article.

This site requires a subscription or purchase to access the full text of books or journals.

Do you have any questions? Contact us.

Or login to access all content.