Interpreting and Authoring Digital Multimedia Narratives
Edited By Len Unsworth and Angela Thomas
Chapter 2: Using Contemporary Picture Books to Explore the Concept of Intermodal Complementarity
← 22 | 23 → CHAPTER TWO
Multimodal literary texts are recognized as a prominent textual form in the landscape of children’s literature. Whether it is a picture book, film, animation, computer game, or digital fiction, children experience these multimodal forms as part of their everyday encounters with literature. Curricula such as the Australian Curriculum: English (ACARA, 2012) have foregrounded the role of multimodal texts within the field of study of literature in English, signifying their importance in contemporary classroom contexts.
A multimodal text is a text that consists of more than one mode or meaning-making system. Modes might include the following: linguistic (words), visual (images, including moving images), audio (sound), gestural (gesture), and spatial (space) (Cope & Kalantzis, 2009). A picture book consists of two modes: linguistic and visual. A film consists of three modes: linguistic, visual, and audio. A piece of digital fiction typically consists of four modes: linguistic, visual, audio, and spatial. A virtual world includes five modes: linguistic, visual, audio, spatial, and gestural.
As teachers explore multimodal texts with children, it is not only the meaning-making potential of each mode that is important, but it is also how those modes work together that is significant. The ways in which individual modes work together is termed intermodality or “intermodal complementarity” (Painter & Martin, 2011). An ideal way to explore the concepts of intermodality is to begin with picture books, where two modes only are at play together: the linguistic and the visual. This chapter...