Time, Truth, Tradition
Edited By András Benedek and Ágnes Veszelszki
The authors outline the topic of visuality in the 21st century in a trans- and interdisciplinary theoretical frame from philosophy through communication theory, rhetoric and linguistics to pedagogy. As some scholars of visual communication state, there is a significant link between the downgrading of visual sense making and a dominantly linguistic view of cognition. According to the concept of linguistic turn, everything has its meaning because we attribute meaning to it through language. Our entire world is set in language, and language is the model of human activities. This volume questions the approach in the imagery debate.
Digital and Visual Literacy: The Role of Visuality in Contemporary Online Reading (Krisztina Szabó)
1. Introduction and Background
Discussing the notion of contemporary online reading in the new media age, it is clear that the shift from printed to online is not just a change of instruments or data media, but also text comprehension. In connection with the latter, we usually come up against the problem of visuality. The line between textuality and visuality seems to blur, their traditional hierarchical order is gradually disappearing, and the problem of reading literacy and its surveys become more complicated. It is obvious that the enormous amount of visual elements which turn up in almost every online text have a significant role in the reading processes. But what is this “significant role”?
In my paper, beginning with this question, I will focus on the role of visual elements in online texts. I will state that visual elements have at least an equal or even dominating role over text in the online reading processes.
In the time of the so-called new media age, many questions are raised concerning the constantly changing way of reading, originating from the issue of the digital devices that have come into use, that can easily substitute for printed books and newspapers. While there are debates and predictions about the future of books and reading itself, more and more attention is being turned to the nature of reading literacy (Ulin 2009; Fekete–Hegedüs–Kis 2010; Cull 2011; Rich 2008; Dougherty 2011). Researchers have the assumption that new types of reading devices contain new types of texts that needs new types of reading skills and methodology (Smolin–Lawless 2003; Coiro–Dobler 2007; Walsh 2010; Murnane–Sawhill–Snow 2012). The shift from printed to digital is so significant that it is better to narrow the focus and talk about digital literacy as a new field of understanding reading.
However, “digital” is not the only essential feature of the issue, but also the online space as well, that lets us wander from one text to another throughout a web ← 103 | 104 → of links. The shift from printed to online is not just a change of instruments or data media (like the ones from papyrus to paper, scroll to newspaper and book, or book to e-book), but also a significant change of comprehending texts, hence a distinction has to be made between digital (offline) and online reading. During online reading, we meet special and, in some ways, new types of text, a kind of visualized hypertext. The line between text and visuality, or text and pictures, seems to blur, and their traditional hierarchical order (in texts the written enjoys priority over pictures) is disappearing (Kress 2003; Youngs–Serafini 2011). Thus, in this paper I am focusing on the role of visuality in contemporary online reading. I state that visual elements, especially pictures, are neither just illustrations nor just explanatory additional elements, nor secondary qualities beside texts, but they have at least an equal, or even dominating role, over text in the online reading process.
In order to confirm my hypothesis, I will discuss the complex connection between online and offline reading and visuality, showing the role of images in the reading process and focusing especially on online reading. My aim with this research in the long run is to help create and develop online reading literacy surveys of PISA (Programme for International Student Assessment), by making the methodological framework more adequate. In 2018, reading literacy will be the major domain of the surveying process, till then both the actual framework and terms should be updated.2
2. About the Complex Connection between Reading and Visuality
Before turning to the role of visual elements in the online reading processes, we should examine the issue of offline reading, too. The whole topic of literacy tradition, from picture reading to picture books, comic books, illustrated workbooks, etc., is so diversified and concerns such a wide range of scientific fields that it goes beyond the scope of this essay. The present focus is on the nature of text itself that originally has a sort of visuality. The “[…] name for a group of signs is text – a collection of signs which organized in a particular way to make meaning. The meanings made will depend on which signs are brought together, and how they are arranged in relation to one another” (Schirato–Webb 2004: 8). The fact that reading is to comprehend signs – which are visual elements with special characteristics such as font style, size and arrangement – means that the whole reading process is subordinated to visuality (Figure 1 and 2). ← 104 | 105 →
Even so, normally we think on the contrary, and make a distinction between text built up by words, built up by letters and other additional visual elements (for e.g. pictures). We are proud to say that we are over the ages of picture-reading and we do not need any visual help for comprehending a text (see the practice of European education starting from recognizing pictures [signs] and aiming to read long texts fluently; Youngs–Serafini 2011; Adamikné 2006; W4). In this sense visual elements only illustrate texts, and complete and explain the meaning of words, if necessary. If it is not, never mind: a well-written text is thought to be understood without any visual support. Thus, in traditional reading, a literacy survey’s text is at the centre, and readers should be able to separate the text (to be read as a task) from its context and comprehend it on its own sake.
This process becomes much more problematic if we would like to survey the reading methods of picture books, workbooks or comics – in these cases we deal with multimodal texts. It means a triple signal system: written language/text, picture/visual elements and a “third quality” (Youngs–Serafini 2011; Szabó 2015b), the latter connecting the two previous levels. In these cases visual elements give meaning and drive the reader’s attention, so there is an interplay between texts and visual elements (Szabó 2015c). At this point, the clear distinction between text and context, text and visual elements fails, and the problem of reading literacy and its surveys become more complicated. Now, suffice to say that the role of visuality in reading literacy is quite important even in the case of offline texts. In the next section I will turn to online texts and the role of visuality in the online reading process.
3. Online Reading and Visuality
In online reading the online text itself is visual. But this implies there to be a different and more complex phenomenon compared to the case of printed texts. An online text is built up from characters (which we can also call signs) and “multiple communication modes” visual elements, for instance image, audio and video. They “are governed by distinct logics [which] change not only the deeper meaning of textual forms but also the structure of ideas, of conceptual arrangements, and of the structures of our knowledge” (Kress 2003: 16, in Weasenforth 2006: 1). This statement underpins the necessity of my previous distinction between digital and online reading: the latter means not just a medium-shift but also a text-comprehending issue. “The world told is a different world to the world shown” (Kress 2003: 1). Writing is for “to tell” while visual elements (for instance images) are for “to show” (Kress 2003). The main difference between them is that “[…] writing […] is governed by the logic of time and by the logic of sequence and its elements in time, in temporally governed arrangements” (Kress 2003: 1). ← 106 | 107 → Contrarily, “the organization of image is governed by the logic of space and by the logic of simultaneity of its visual/depicted elements in spatially organized arrangements” (Kress 2003: 1). However, I must disagree – even in the special case of printed picture books, workbooks and comics. Take the latter as an example: in comic books, pictures are also governed by the logic of time, they are in temporally governed arrangements whereas they are also governed by the logic of space. And the same phenomenon could be easily noticed vice versa. The issue is the same in the case of the online version of picture books, workbooks and comic books (Youngs–Serafini 2011). Take an average text on the Internet rather than these specific types of texts. The label “average” is problematic; there are debates on the question of what it means to be a “text” in the online space (Cull 2011; Bolter 2001; Szabó 2015c; Dyson–Kipping 1998). This vagueness is rooted exactly in the nature of visuality that interweaves the online contents. An online text is more than a digitalized version of an offline text, because the online nature of it essentially modifies its reading, meaning and comprehension. Researchers call these “hypertexts,” which are “linked to each other with hyperlinks so we can easily switch and jump between them, like in a kind of eternal, never-ending and always refreshed text” (Szabó 2015c: 171). Hypertexts, also because of the online space, naturally “live together” with visual elements. This connection could be so complex that sometimes it is difficult to decide what is related to the main text, and what is just an additional illustrative or design element or a supporting icon of the digital device. This could be a problem for instance in online literacy surveys: if we do not have exact notions about the texts that are to be read, then we will easily conclude inappropriate results about readers’ skills and the reading process as well.
However, this should not be too surprising: if the connection between an offline text and visuality is so strong and complex, then it should be at least the same in the case of online texts, too. It is vital to emphasize that the observation of online texts should start from the thought which discusses texts and visual elements as interplay and a sort of interlock. Thus, texts and visual elements in the online space are at least not subordinated but they supplement each other. As we will see in the next section, visual elements have a much stronger and inevitable role in online texts and online reading comprehension.
4. The Special Role of Visual Elements in Contemporary Online Reading
Researchers investigating the nature of online text comprehension regularly indicate the following observations. First, defining online text is problematic because of the strong presence of visuality. Second, visual elements significantly influence ← 107 | 108 → and modify the meaning of a text, thus we should examine the mutual effects of texts and visual elements on each other in online reading processes (Szabó 2015c). Above, I have already shown that visual elements, both in the case of printed and online texts illustrate, complete and explain the main text – on the first level of comprehension. Furthermore, they give meaning to the texts and drive the reader’s attention during the reading process. In short, there is interplay among the text and visual elements. Now, I claim that visual elements have at least an equal or even a dominant role over text in the online reading processes. Visual elements do not just support comprehension, but dominate or even replace online texts. “[R]eaders’ imagination is a matter of ordering elements in contrast to filling traditional text with meaning. […] readers do fill images with meaning also” (Weasenforth 2006: 27). Moreover, “images are plain full of meaning, whereas words wait to be filled” (Kress 2003: 3). Thus, “the conceptualization of textuality is changing as images seem to dominate text and as screen overtake paper […]” and “writing becoming subordinated to the logic of the visual” (Kress 2003: 5). This is a cardinal change in comparison to traditional theories about printed texts, claiming the dominance of texts over visual elements. In this sense we could talk about the rediscovery of the visual.
It is time to reflect on the topic called visual literacy. Visual literacy is the “ability to decode, interpret, create, question, challenge and evaluate texts that communicate with visual images as well as, or rather than, words. Visually literate people can read, interpret the purpose and the intended meaning, and evaluate the form, structure and features of the text. They can also use picture and word images in a creative and appropriate way to express meaning” (Carry n. a.: 13). This definition is based on a theory implying that “picture books, magazines, newspapers, advertisements and films create meaning through shape, line, colour, actions, eye paths and angles” (W3: 3). Thus, here again, the main issue is the role of visual elements embedded into some kinds of texts. The connection between text and visual is so strong that in some cases – e. g. in online reading – the text is subordinated to the visual. To be “[…] able to make sense of what we see in images and understanding how these pictures relate to the writing associated with them” (W3: 2) is the main issue of visual literacy – and online reading literacy, as well.
There is continuity in the printed – digital – online reading triad and visuality has its own important, variant and intensifying role in every step of this path. Focusing on online reading, the special role of visual elements could be the following (Figure 3). ← 108 | 109 →
(source: image created by the author)
With this we have returned to the question of what it means to be “text” in the online space. With taking the notion of visual text we get an even more complex picture, where we should take the following features into consideration as well (Figure 4).
(source: Carry n. a.: 8)
According to the above, the answer would be: online text is like the web itself. It is a sort of content composed from text (characters, words, sentences) and visual elements (pictures, images, graphics, etc.). With this definition, the already fuzzy concept of online text comprehension may become even more complex, but it could serve as a step forward to get a better understanding on online reading literacy. ← 109 | 110 →
In my paper I focused on the special role of visuality in the online reading process. I claimed that visual elements are neither just illustrations, nor just explanatory additional elements nor secondary qualities beside texts, but they have at least an equal or even a dominating role over text in online reading processes. First, to support my hypothesis, I separated the notions of printed, digital and online reading from each other and then I examined the connection between offline reading and visuality in the traditional sense. Second, I discussed the role of visual elements in the case of printed text and online texts as well as visual texts, and referred briefly to some special cases, too. I demonstrated on three levels that visual elements have three important roles in online texts and – contrary to traditional views – texts do not dominate the visual anymore. Moreover, visuality has gained dominance over text in the online space, and it is better to talk about texts as webs or contents which built up from characters and visual elements.
This shift in the hierarchy of texts and visual elements is so significant that it basically modifies and influences online reading and text comprehension – thus we should take it into account in contemporary reading literacy research and surveys as well.
Adamikné Jászó, Anna (2006): Az olvasás múltja és jelene. Budapest: Trezor Kiadó.
Bolter, Jay David (2001): Writing Space: The Computer, Hypertext, and the Remediation of Print. Second edition. Mahwah, NJ, London: Lawrence Erlbaum.
Carry, Diana Dumetz (n. a.): Visual Literacy: Using Images to Increase Comprehension. https://readingrecovery.org/images/pdfs/Conferences/NC09/Handouts/Carry_Visual_Literacy.pdf.
Coiro, Julie – Dobler, Elizabeth (2007): Exploring the Online Reading Comprehension Strategies Used by Sixth-Grade Skilled Readers to Search for and Locate Information on the Internet. Reading Research Quarterly 42/2: 214–257.
Cull, Barry W. (2011): Reading Revolutions: Online Digital Text and Implications for Reading in Academe. First Monday 16/6. http://firstmonday.org/ojs/index.php/fm/article/view/3340/2985.
Dougherty, William C. (2011): The Book is Dead, Long Live the Book! Managing Technology. http://www.sciencedirect.com/science/article/pii/S0099133311001959.
Dyson, Mary Clare – Kipping, G. J. (1998): Exploring the Effect of Layout on Reading from Screen. In: Hersch, Roger D. – André, Jacques – Brown, Heather (eds.): Electronic Publishing, Artistic Imaging, and Digital Typography. Seventh ← 110 | 111 → International Conference on Electronic Publishing: Proceedings. Berlin: Springer. 294–304.
Fekete, Márton – Hegedüs, Nikolett – Kis, Andrea (2010): A megváltozott olvasás. Szkholion 2010/2: 72–80.
Kress, Gunther (2003): Literacy in the New Media Age. London: Routledge.
Murnane, Richard – Sawhill, Isabel – Snow, Catherine (2012): Literacy Challenges for the Twenty-First Century: Introducing the Issue. The Future of Children 22/2: 3–15.
Rich, Motoko (2008): The Future of Reading. Literacy Debate: Online, R U Really Reading? Series-NYTimes.com. http://www.nytimes.com/2008/07/27/books/27reading.html.
Schirato, Tony – Webb, Jen (2004): Reading the Visual. Crows Nest: Allen & Unwin.
Smolin, Louanne Ione – Lawless, Kimberly A. (2003): Becoming Literate in the Technological Age: New Responsibilities and Tools for Teachers. The Reading Teacher 56/6: 570–577.
Szabó, Krisztina (2014): A digitális szövegértés felmérésének kihívásai. Megfontolások a PISA-tesztek módszertani fejlesztésével kapcsolatban. In: Csiszár, Imre – Kőmíves, Péter Miklós (eds.): DOSZ. Tavaszi Szél 2014 Konferenciakötet. IV. Debrecen: Doktoranduszok Országos Szövetsége. 429–438.
Szabó, Krisztina (2015a): A digitális szövegértés felmérése. A 2009-es PISA-teszt definíciós problémái. In: Schaub, Anita – Szabó, István (eds.): IDK 2014 Pécs Konferenciakötet. Pécs: Pécsi Tudományegyetem Doktorandusz Önkormányzat. 537–548.
Szabó, Krisztina (2015b): Digital Literacy: Is Digital Reading Similar to Comic Reading? Opus et Educatio 2/1: 48–56.
Szabó, Krisztina (2015c): Digital Reading and Text Comprehension: Comic Reading as a New Metaphor for Digital Reading. In: Beseda, Jan – Machát, Zbynek (eds.): DisCo 2015. From Analog Education to Digital Education. 10th Conference Reader. Prague: Center for Higher Education Studies. 167–178.
Ulin, David L. (2009): The Lost Art of Reading. Los Angeles Times. http://articles.latimes.com/2009/aug/09/entertainment/ca-reading9.
Walsh, Maureen (2010): Multimodal Literacy: What Does it Mean for Classroom Practice? Australian Journal of Language Literacy 33/3: 211–239.
Weasenforth, Donald (2006): Review of Literacy in the New Media Age. Language Learning & Technology 10/2: 25–28.
W1 = A sample list of fonts installed with Microsoft Windows Vista and Windows 7. http://ptgmedia.pearsoncmg.com/images/chap3_9780789749550/elementLinks/03fig23_alt.jpg (Figure 1).
W2 = Example for font-arrangement. http://mrfletcher.weebly.com/shape-poems.html (Figure 2).
W3 = Every Picture Tells a Story. Accessing Visual Text. https://dei.eduone.net.au/projectresources/media/AccessingVisualTextFinal.pdf.
W4 = Módszertani kézikönyv a magyar nyelv tanításához az 1. osztályban. 2009. http://www.szmpsz.sk/szmpsz/hir/2009/0902_abeceskonyv/metodika.pdf.
1 This ongoing research is conducted in the framework of Integral Argumentation Studies, OTKA – K-109456 at the Doctoral School of Philosophy and History of Science, Budapest University of Technology and Economics.
Special thanks go to János Tanács PhD, István Danka PhD and all colleagues at the Department of Philosophy and History of Science, Budapest University of Technology and Economics, who helped me with their professional advice during my research.
2 For more details and critics about the PISA-subject see my previous articles: Szabó 2014; 2015a.