Festschrift in Honour of Günther Sigott: Advanced Methods in Language Testing

Acknowledgements

Besides the academic tribute it pays, this volume is first and foremost intended as very much a personal ‘thank you’ to a friend, mentor, and colleague, for all that he did for many of us, always going ‘the extra mile’ above and beyond the line of duty. Therefore, first and foremost, thanks are due to Günther, for being the kind of person and scholar that he is. Then, a huge “thank you” goes out to all the contributors, for sharing their research and thoughts with us and Günther. We would also like to thank other friends of Günther, who supported this Festschrift and would have liked to contribute were it not for different constraints. These include Ute Knoch, Monique Reichert, Khaled Barkaoui, Gertraud Havranek, Tim McNamara, Tom Lumley, Adrian Palmer, and Luke Harding. Thanks is also due to the publisher, Peter Lang, and Michael Rücker for the dedicated support with the technical aspects of the publication. Finally, gratitude is also owed to Universität Klagenfurt and the Faculty of Arts, Humanities and Education for generously funding the volume at hand.

Introduction

The volume at hand, traditionally called a Festschrift in the German-speaking world, has been compiled to honour Günther Sigott for his notable contributions to academia in general and to the field of language testing in particular. It is no exaggeration to say that Günther Sigott was one of the key figures placing language testing and assessment on the academic map in Austria. His work within the national testing standards and the manner in which this work communicated the importance of language testing also to the public helped solidify the relevance of the discipline. In this respect, Günther paved the path for many of us working as language testing scholars in Austria at a time when language testing was not yet established as a scholarly field outside the English-speaking world. Günther was in fact the first linguist in Austria to be “habilitiert” with a venia docendi¹ in applied linguistics. Further hallmarks of his career are his editorship of the Language Testing and Evaluation series at hand (counting 48 volumes to date, including this one), his ground-breaking work on establishing national standards for the English Matura (and additional work on German and classical languages), the establishment of the Klagenfurt Language Testing Center, the running of a Master’s degree in language testing, and the countless collaborations with scholars nationally and internationally. The breadth and depth of his research interests, with a particular focus on language assessment, is sketched out in this volume. It ranges from the broad interest in validity as a fundamental principle permeating his work, and measurement aspects in general, to the very specific interest in C-tests, which constituted a significant part of Günther’s research.

Günther started on his academic path by enrolling in a teaching degree in English and French at Graz University in 1978, motivated by his interest in languages, which was sparked early on by exposure at home (his father being a manager of an internationally operating company). It was this wish to be a language teacher that has defined Günther’s work and interests ever since. During his probationary year as a grammar school teacher in Graz, he was faced with the question of whether a test he was about to implement in the classroom was actually valid or not. The pursuit of finding an answer to this question, in various contexts, became a leitmotif of Günther’s career.

Günther made his first steps in academia at Graz University in 1984, working as an Assistant at the Department of Romance Languages (specializing in French). It is, therefore, no surprize that his PhD, obtained at the Klagenfurt University in 1991, focused on the learnability of French and English. He took up a permanent post at Klagenfurt University in 1987, this time at the Department of English. The department was at the time directed in all matters linguistic by William Nemser, steering it strongly towards second language acquisition as the main research theme. This is where Günther’s interest started to focus on assessment and where his endeavour began to put language assessment on the linguistic agenda in Austria.

Günther managed, in the period between the early 1990s and early 2000s, to establish quantitative approaches in linguistic studies, along with them the measurement of language competence, as a legitimate calling for a linguist. This culminated in Günther’s Habilitation in Applied Linguistics in 2003. While this was the first ever Habilitation in Applied Linguistics in Austria, many more have followed Günther’s path.

Günther’s work in his mid-career is best characterized by his dedication to studying language learning, i.e., what many consider to be Applied Linguistics in the core sense of the term. In Günther’s words, delivered at his Habilitation lecture in 2003, “[I]‌ consider Applied Linguistics as the scientific study of phenomena inherent in and surrounding the activities of language learning and language teaching as well as the application of principles and concepts from Linguistics and other fields in order to support these activities”. Günther saw language assessment as one of the central dimensions of language learning and teaching, and dedicated his work on his Habilitation to C-tests, which are highly reliable and economical instruments to measure general language proficiency. Günther examined the operational side of the C-test as well as the underlying construct, searching for indicators of the instrument’s validity.

This period is marked by multiple (sole authorship and collaborative) publications focusing on experimental runs of C-tests with various parameters being varied, from participants to types and kinds of deletions, to different languages being involved.

As language testing was already a firmly established scholarly field in the English-speaking world, it comes as no surprise that Günther spent time in Lancaster in the early 2000s. The experience with the UK language testing community not only helped Günther form his own research interest, but it also resulted in the founding of the Language Testing and Evaluation series at hand, established in 2004. The series took its starting point during a talk between Günther and Rüdiger Grotjahn, who was a professor at the Ruhr-University Bochum with a keen interest in language testing and C-tests. Günther’s Habilitationschrift eventually became the first volume in the new series. The two editors were later joined by Claudia Harsch as the third series editor, and meanwhile, Rüdiger Grotjahn took his well-deserved retirement. Over the past 20 years and with 47 volumes, the series has provided a forum for theoretical and empirical research in a wide range of aspects related to language testing and evaluation.

After his Habilitation, Günther worked on establishing standards for the first national examinations in English (i.e., the Matura). The project with the Ministry of Education began in 2006 and, spanning almost a decade, resulted in the creation of the Klagenfurt Language Testing Center. The Center still serves as a seminary for language testers in Austria, and it periodically offers a postgraduate (MA) degree in language testing, met by great interest from the teaching and academic community. Based on his expertise, Günther became a consultant for the German Abitur, as well as for the national exams in classical languages in Austria. This period in Günther’s career resulted in Volume 40 of this series, taking stock of language testing in Austria (2018).

Günther’s main research subject has always been language learning, in all its forms and varieties as a field of applied linguistics, including aspects such as measurement, feedback provision, and error analysis. His list of publications and presentations, both nationally and internationally, is impressive, and his plans for retirement show no sign of slowing down. We can expect further work from Günther, and there are even rumours as to a text book on language assessment literacy being prepared. In that respect, the above outline of Günther’s career does not intend to mark the end of Günther’s work, but rather a summary and celebration of what he has achieved so far. As many of Günther’s Austrian colleagues had contributed to the aforementioned 2018 volume, the current Festschrift takes a more international stance, compiling contributions from colleagues and friends around the world to honour Günther’s work.

The contributors to this volume come from all around the world, spanning regions from Austria and its neighbouring countries Slovenia and Germany, to the UK, USA, and all the way to Japan and Iran. They work as test developers, teacher educators, psychometricians and researchers, taking up a variety of themes such washback of standardized tests, the acknowledgement of language varieties in high-stakes exams, fostering language assessment literacy or psychometric models to enhance C-test interpretations or scoring validity. As diverse as their takes on language assessment-related aspects may be, the authors have several things in common. First, they have, in different functions and constellations, collaborated or worked with Günther over the years; second, they all share Günther’s curiosity and interest in ‘digging’ deeper into yet unknown aspects and concepts, be it in the realm of C-tests and their constructs, or be it the role that grammatical accuracy plays in a given exam; a third commonality is their concern for validity, their strive for examining and where possible enhancing the validity of specific language tests or generic aspects of language assessment. Validity is, therefore, the main red thread throughout this volume, which constitutes a rich bouquet to celebrate Günther’s academic career and achievements.

Focusing on the theme of validity, Barry O’Sullivan and Micheline Chaloube-Deville set the scene with a much-appreciated integrative view on validity, bringing different schools of thought and divergent traditions from the UK and the USA together. They take a very promising look on validity, which they argue needs to span four dimensions, that is, test development, measurement theories, consequences, and communication with all stakeholders. The latter two dimensions are ever so important, as they focus on the impact tests may have on language learners and their social or educational environments, as well as on the wider realm of stakeholders and test users. Communicating test purposes and intended uses with these stakeholders is a crucial desideratum, as it can contribute towards a more appropriate and hence more valid use and interpretation of test results. All chapters in this volume connect in one or more ways to this integrated view on validity and contribute towards enhancing validity efforts in one way or another. Hence, Barry O’Sullivan and Micheline Chaloube-Deville’s paper is a very fitting opener for Günther’s Festschrift.

With the second chapter, we will stay in the realm of O’Sullivan and Chaloube-Deville’s fourth dimension, that is, communication with stakeholders, and give Klaus Siller space to develop his argument for bringing test tasks from a large-scale standardised test, used for educational monitoring, into the language classroom. This way, he aims to open communication channels between test developers on the one hand and teachers and learners on the other hand. His focus lies on employing reading tasks in such a way as to foster reading abilities rather than ‘only’ measuring them. Based on results from a pilot study that revealed what learners actually do when working on the reading tests, Klaus Siller develops practical suggestions on the use of these tasks in the classroom. Giving teachers insightful ideas on how to best make use of existing reading tasks is one way of empowering them and enhancing their language assessment literacy (LAL).

Staying in Austria and in the realm of LAL among language teachers, the third chapter takes a closer look at what is needed to develop LAL with teacher trainees, one prerequisite to enable future teachers to meaningfully develop, employ and interpret tests and assessments in their classrooms. Armin Berger and Helen Heany conducted a survey in order to find out how challenging different known components of LAL are perceived by pre-service teachers. They then used their to design a LAL course at the University of Vienna, taking the perceived difficulties of different LAL elements into consideration in terms of progression of both theoretical as well as practical elements. Such training can contribute towards enabling teachers to develop, administer and employ valid language assessments, as well as interpret them in a valid way, in order to foster learner development.

While chapter three looked at language assessment literacy and teacher training, we are now moving to a high-stakes language test at the end of secondary schooling, which also entails the right to enter university, albeit in one specific country, i.e., Slovenia. Gašper Ilc and Andrej Stopar zoom in on language accuracy as one sub-component of language proficiency and examine its relation to other parts of the exam. Their findings show that language accuracy, while having moderate to strong correlations with the reading section in particular, nevertheless has its own place in the exam, as they found only moderate correlations between the different test parts that target accuracy. Such studies serve to examine the validity of different exam parts with regard to their shared construct, both with regard to the exam’s design and to measurement aspects.