Loading...

Developments in Glocal Language Testing

The Case of the Greek National Foreign Language Exam System

by Evdokia Karavas (Volume editor) Bessie Mitsikopoulou (Volume editor)
©2019 Edited Collection XXIV, 364 Pages

Summary

In contrast to most publications in language testing and assessment, which address research and developments of international, monolingual exam batteries, this volume aims to present research studies focusing on a locally developed multilingual exam suite known as KPG (The Greek National Foreign Language Exam System). The KPG exams, which represent a «glocal» multilingual examination battery, are the first of their kind in Europe, and take into account local needs, global conditions of knowledge and production, and international concerns regarding testing and assessment.
The chapters included in this volume highlight many of the innovative features of the KPG exam suite and focus on various issues relating to the reliability and validity of glocal language tests. These include the effect of text and reader variables on reading comprehension, different world representations and ideologies in reading texts, interlocutor performance variability, source text regulated written mediation performance, writing and listening task difficulty, and the effect of task and assessment variables and corpus-based research of text grammar on KPG candidates’ scripts – all of which have important implications for global language testing and practices.

Table Of Contents

  • Cover
  • Title
  • Copyright
  • About the author(s)/editor(s)
  • About the book
  • This eBook can be cited
  • Contents
  • List of Figures
  • List of Tables
  • List of Abbreviations
  • Foreword (Elana Shohamy)
  • Preface
  • Introduction Issues and challenges in glocal language testing: The case of the Greek State Certificate in Language Proficiency (Evdokia Karavas / Bessie Mitsikopoulou)
  • The KPG system: A glocal multilingual examination suite
  • Additional innovatory features of the KPG exam system
  • Researching the KPG English exam
  • Conclusion
  • Bibliography
  • Appendix: Introducing the KPG as a national project
  • Part I Understanding and producing oral texts
  • 1 The sociosemiotic language view underlying the design of the KPG exams (Bessie Mitsikopoulou / Evdokia Karavas / Christina Lykou)
  • A sociosemiotic view of language
  • The impact of the sociosemiotic approach on test tasks
  • A genre-based approach to writing
  • Conclusion
  • Bibliography
  • 2 Oral text difficulty in foreign language listening comprehension testing (Elisabeth Apostolou)
  • Introduction
  • Background to the study
  • Linguistic variables affecting oral text difficulty: Syntactic complexity and lexical difficulty
  • Paralinguistic variables affecting oral text difficulty: The role of speech rate and accent
  • Cognitive variables: The role of background knowledge
  • Research methodology
  • Quantitative data: Candidates’ questionnaires
  • Qualitative data: Interviews and verbal reports
  • Presentation of findings and discussion
  • Candidate questionnaire data
  • Learners’ verbal reports
  • Discussion and conclusion
  • Bibliography
  • 3 The impact of oral examiners’ conduct on candidates’ performance in the KPG exam (Xenia Delieza)
  • Introduction: Oral language assessment and the KPG oral exam in English
  • Literature review
  • The examiner variable in oral language proficiency testing
  • Background of the study
  • The KPG speaking test
  • The Oral Examiner Training Programme
  • The Observation Project
  • Purpose of the study and methodology
  • Data collection and analysis
  • Findings: Types of examiner involvement: Quantity and quality
  • Results from observation
  • The B2 level oral exam
  • Results from simulated tests
  • Comparisons and conclusions
  • Discussion and implications
  • Conclusion
  • Bibliography
  • Appendix I
  • Appendix II
  • Part II Understanding written texts
  • 4 Investigating ideologies in texts used in global and ‘glocal’ language testing (Amalia Balourdi)
  • Introduction
  • Literature review
  • Ideology and discourse
  • Transitivity analysis in SFL
  • Purpose of the study and methodology
  • Quantitative analysis
  • Qualitative analysis: The construal of three different realities around ‘giving’ and ‘taking’
  • Conclusion
  • Bibliography
  • 5 Visual matters in language testing: Exploring intersemiotic meaning making in reading comprehension source texts (Styliani Karatza)
  • Introduction
  • SFL and image/text relations in terms of content: Intersemiosis
  • Investigating intersemiotic visual–verbal relations through Systemic Functional Multimodal Discourse Analysis (SF-MDA)
  • Findings about visual-verbal relations of KPG reading comprehension source texts
  • Analysis of gist items
  • Discussion about ideationally oriented intersemiotic literacy requirements and implications
  • Conclusion
  • Bibliography
  • 6 Test-takers’ strategies in responding to KPG English language reading comprehension tasks: Evidence from retrospective think-aloud protocols (Trisevgeni Liontou)
  • Introduction
  • Literature review
  • Research methodology
  • Findings
  • Item type classification
  • Strategy use in main idea items
  • Strategy use in factual information items
  • Discussion
  • Conclusion
  • Bibliography
  • Acknowledgements
  • Appendix
  • Part III Producing written texts
  • 7 A comparative study of cohesion in L2 candidates’ texts (Virginia-Maria Blani)
  • Introduction
  • Literature review in L1 and L2 research on cohesion
  • Research context, data and methodology
  • Corpus
  • Methodology
  • The adopted model for the cohesion analysis
  • Data analysis procedure
  • Presentation of the results and discussion
  • Grammatical cohesion
  • Conjunctions
  • Reference
  • Lexical Cohesion
  • Conclusion and implications
  • Bibliography
  • 8 The effect of source text on read-to-write task difficulty (Vasso Oikonomidou)
  • The aim and context of the study
  • Read-to-write tasks in foreign language exams
  • Research on read-to-write task difficulty
  • Source text related factors affecting read-to-write task difficulty
  • Research methods: Two types of analysis
  • Aim and research questions
  • The data: B1 level source texts and task scores
  • Methods of data analysis
  • Research findings
  • Source text analysis
  • Scoring analysis
  • Discussion of findings
  • Conclusions and implications
  • Bibliography
  • Appendix
  • 9 Assessing cross-language mediation in the Greek national multilingual exam suite: Test-takers’ language performance and strategy use (Maria Stathopoulou)
  • Introduction: The context of the study
  • Cross-language mediation in foreign language didactics
  • Mediation: A neglected area?
  • Towards an understanding of the notion of cross-language mediation
  • Mediation tasks and mediation strategies
  • Aim of the study
  • Data collection methods
  • Presentation and discussion of results
  • The popularity and frequency of mediation strategies in successful and less successful scripts
  • Conclusion
  • Bibliography
  • Appendix I
  • Appendix II
  • Part IV Electronic developments
  • 10 Documenting language proficiency levels using learner data from the KPG examinations suite (Voula Gotsoulia)
  • Enriching language proficiency descriptions
  • A multilingual database with descriptions of levelled language competences
  • Language performance data
  • The representation of interrelated language competences
  • A lexical semantic model for corpus annotation
  • Illustrative corpus analyses for English
  • Conclusion
  • Bibliography
  • 11 KPG e-school: The diffusion and implementation of an educational innovation (Bessie Mitsikopoulou / Evdokia Karavas / Smaragda Papadopoulou)
  • Introduction: Setting the context
  • Designing the KPG e-school
  • Strategy apps in the students’ electronic class
  • Purpose
  • Structure
  • Design and development process
  • Specifications, principles and features
  • E-books and activity database in the teacher’s electronic class
  • Purpose
  • Structure
  • Design and development process of the activity database
  • Specifications, principles and features
  • The parent’s electronic class
  • Purpose
  • Structure
  • Conclusion
  • Bibliography
  • Conclusion (G)Local alternatives in testing (Evdokia Karavas / Bessie Mitsikopoulou)
  • Localised language testing
  • Glocal language testing
  • Bibliography
  • Appendix
  • The structure and content of the KPG exams
  • A level exam
  • B level exam
  • C level exam
  • Notes on Contributors
  • Index
  • Series index

| ix →

Figures

Figure 5.1: TEXT: Workaholic

Figure 5.2: TEXT: Politeness

Figure 5.3: TEXT: satellites

Figure 5.4: TEXT: women’s history

Figure 5.5: TEXT: Are we alone

Figure 5.6: TEXT: Pay attention

Figure 11.1: The reading strategy app for KPG exam in English (B level)

| xi →

Tables

Table 2.1: Questionnaire survey sample

Table 2.2: Subjects’ profile

Table 2.3: Questionnaires’ high difficulty rates

Table 3.1: The content and structure of the KPG B2 and C1 speaking tests

Table 3.2: Frequencies of examiner intervention per test task in the B2 speaking test in May 2007 and November 2007 observation phases

Table 3.3: Frequencies of examiner intervention per test task in the C1 level speaking tests in May 2007 and November 2007 observation phases

Table 3.4: Number of instances of involvement in the two levels per Activity

Table 3.5: Types of involvement as identified in the simulated speaking tests and percentages for each Activity

Table 3.6: List of acceptable versus non-acceptable intervention

Table 3.7: Percentages of different types of changes of rubrics in the B2 and C1 oral test in May

Table 3.8: Percentages of ‘other’ types of changes of rubrics (in the B2 and C1 oral test in May 2007) classified as acceptable and non-acceptable

Table 3.9: Percentages of different types of interruption of candidates in the B2 and C1 oral test in May

Table 3.10: Percentages of ‘other’ types of interruption of candidates (in the B2 and C1 oral test in May 2007) classified as acceptable and non-acceptable

Table 4.1: Types and sub-types of material processes across the three sub-corpora ← xi | xii →

Table 4.2: Frequencies of human actants across the three sub-corpora

Table 4.3: Frequencies of non–human actants across the three sub-corpora

Table 5.1: Instances of visual–verbal relations

Table 5.2: Visual–verbal relations: Grouping of data instantiations

Table 5.3: Types of Gist Items (TGIs)

Table 6.1: Sample of KPG reading comprehension test item types

Table 6.2: Strategies for correct responses to main idea items

Table 6.3: Strategies for incorrect responses to main idea items

Table 6.4: Strategies for correct responses to factual information items

Table 6.5: Strategies for incorrect responses to factual information items

Table 6.6: Reading strategies coding system (adapted from Cohen and Upton 2007)

Table 6.7: Test-taking strategies coding system (adapted from Cohen and Upton 2007)

Table 7.1: Grammatical cohesion analysis results

Table 7.2: Types of conjunction per script category and statistical significance chi-square test results

Table 7.3: Chi-square significance test results for adversative conjunctions

Table 7.4: Chi-square significance tests results for causal conjunctions

Table 7.5: Types of reference per script category (raw frequency per type) and chi-square test significant difference results

Table 7.6: Raw and relative frequencies of lexical cohesion types

Table 7.7: Chi-square test results for lexical cohesion sub-types

Table 8.1: Text related factors affecting task difficulty

Table 8.2: Source text related factors in this investigation ← xii | xiii →

Table 8.3: Source text complexity indices

Table 8.4: Readability indices of source texts

Table 8.5: Syntactic complexity indices of source texts

Table 8.6: Lexical complexity indices of source texts

Table 8.7: Test taker performance on read-to-write tasks

Table 8.8: Task analysis

Table 9.1: Types of mediation strategies

Table 9.2: Degree of popularity of mediation strategies in fully satisfactory and moderately satisfactory scripts

Table 9.3: Type A and Type B mediation strategies in fully and moderately satisfactory scripts across levels: Frequencies

Table 9.4: Mediation strategy use in fully satisfactory and moderately satisfactory scripts across levels: Frequency

Table 9.5: Mean number of words used to realise mediation strategies

Table 9.6: Mediation strategies spotted in the corpus (Stathopoulou 2015)

Table 9.7: Mediation strategy use in fully satisfactory and moderately satisfactory scripts across levels: Frequency

Table 10.1: Language proficiency descriptors linked with lexico-grammatical data

Table 10.2: The Judgment frame

Table 10.3: Output of the Semafor parser

Table 10.4: Lexical coverage of FrameNet (FN) in sample KPG sub-corpus

Table 10.5: Discrete frame occurrences in sample sub-corpus and sentences per script

Table 10.6: Levelled, contextual lexical use in Judgment

Table 10.7: Levelled, contextual use of ‘admire’ in Judgment

Table 11.1: Contents and modalities used in Modules 1 and 4

| xv →

Abbreviations

Adv Adverbial

ANOVA Analysis of Variance

CALT Computer Adaptive Language Testing

CDA Critical Discourse Analysis

CEFR Common European Framework of Reference for Languages

CESOL Cambridge ESOL exams

CLD Curriculum Language Database

CLC Cambridge Learner Corpus

CMS Content Management System

EFL English as a Foreign Language

ELIUM Michigan Language Exams

ELL Early Language Learning

EPP English Profile Programme

ESL English as a Second Language

EU European Union

FAQs Frequently Asked Questions

FL Foreign Language

IELTS International English Language Testing System

IFLC Integrated Foreign Languages Curriculum

IMS Inventory of Mediation Strategies

KPG Kratiko Pistopiitiko Glossomathias

L1 First Language

L2 Second Language

LPI Language Proficiency Interview

MDA Multimodal Discourse Analysis

MMA Multimodal Analysis Image

NP Noun Phrase

OPI Oral Proficiency Interview

PP Prepositional Phrase

RCeL Research Centre for Language Teaching, Testing and Assessment ← xv | xvi →

SFL Systemic Functional Linguistics

TGI Test Gist Item

TSRP Test Taking Strategies Research Project

TWE Test of Written English

V Verb

WCMS Web Content Management System

| xvii →

ELANA SHOHAMY

Foreword

It is a special honour to be asked by Evdokia Karavas and Bessie Mitsikopoulou, the editors of this volume, to write a foreword to this special book in honour of Professor Bessie Dendrinos. The book, entitled Developments in Glocal Language Testing: The Case of the Greek National Foreign Language Proficiency Exam, represents the state of the art in language testing, since it integrates language testing with multiple societal and educational perspectives, that is, the political, the social, language policy, linguistics and psychometrics. This integration is manifested through the case of the Greek National Foreign Language Proficiency Exam, an examination battery that Professor Dendrinos devoted her work to in the past two decades. Thus, the findings reported in the chapters of this book are derived from research and development within the Greek testing system. Yet, the implications of these findings go far beyond the Greek case as they are relevant to the language testing field at large in many worldwide systems. Hence, the word ‘glocal’ in the title implies both the local and the global. The other unique dimension of the chapters is that the research described in the volume was conducted with, and/or under the direction of Professor Dendrinos, with her creative and critical thinking.

I would like to acknowledge and recognise the special contribution that Professor Dendrinos made to the Applied Linguistics field and to my own thinking and academic growth. Bessie is a scholar who affected many researchers and had an influence on their critical views, seeking language justice in relation to English and other languages.

In the mid-1990s, I attended a lecture delivered by Professor Dendrinos at one of the GURT conferences held at Georgetown University in Washington DC, organised by the late Professor James Alatis. I had the privilege of listening to Professor Dendrinos addressing issues which at the time were very new to me and required great courage to discuss. This was an ← xvii | xviii → era when the number of people learning English was growing exponentially, especially in the third-world countries where governments and Ministries of Education were introducing English in school systems, in the belief that knowledge of English would improve the economy and status of their counties. It was during that lecture that Professor Dendrinos delivered the message about the power of the English language and the inequalities it produced, especially in relation to the western world. Bessie demonstrated this in a number of ways, but the one I recall best was how the contents included in the English textbooks gave false ideas to the people of the third world that learning the English language would make them successful in life. She argued that English textbooks portray those who know the English language as wealthy, rich, smart and having endless opportunities and possibilities. Quite often, according to Dendrinos, the English language textbooks equated speakers of English with the ‘haves’, leading the learners in the third world to believe that if they were proficient in the English language, they would be able to attain the same status and wealth.

Details

Pages
XXIV, 364
Year
2019
ISBN (PDF)
9781787076921
ISBN (ePUB)
9781787076938
ISBN (MOBI)
9781787076945
ISBN (Softcover)
9783034322416
DOI
10.3726/b11390
Language
English
Publication date
2019 (January)
Keywords
research in language testing glocal testing Foreign language proficiency examinations
Published
Oxford, Bern, Berlin, Bruxelles, New York, Wien, 2019. XXIV, 364 pp., 7 fig. b/w, 56 tables

Biographical notes

Evdokia Karavas (Volume editor) Bessie Mitsikopoulou (Volume editor)

Evdokia Karavas is Associate Professor in the Department of English Language and Literature of the National and Kapodistrian University of Athens and Deputy Director of the RCeL (Research Centre for Language Teaching, Testing and Assessment). She holds a PhD from the University of Warwick, where she taught for a number of years as lecturer at the Centre for English Language Teacher Education. Her research interests include language teacher education and development, language testing and assessment, curriculum/programme evaluation and implementation research. Bessie Mitsikopoulou is Associate Professor in the Department of Language and Linguistics, Faculty of English Language and Literature, National and Kapodistrian University of Athens. She is the Director of the Centre of Self-Access Learning and Materials Development and she has co-ordinated various projects in the areas of language education, curriculum reform, critical literacies and the production of digital interactive materials for foreign language education. Her research interests are in the areas of critical discourse analysis, applied and educational linguistics, new media and digital technologies in education, critical and academic literacies, and writing theories and practices.

Previous

Title: Developments in Glocal Language Testing
book preview page numper 1
book preview page numper 2
book preview page numper 3
book preview page numper 4
book preview page numper 5
book preview page numper 6
book preview page numper 7
book preview page numper 8
book preview page numper 9
book preview page numper 10
book preview page numper 11
book preview page numper 12
book preview page numper 13
book preview page numper 14
book preview page numper 15
book preview page numper 16
book preview page numper 17
book preview page numper 18
book preview page numper 19
book preview page numper 20
book preview page numper 21
book preview page numper 22
book preview page numper 23
book preview page numper 24
book preview page numper 25
book preview page numper 26
book preview page numper 27
book preview page numper 28
book preview page numper 29
book preview page numper 30
book preview page numper 31
book preview page numper 32
book preview page numper 33
book preview page numper 34
book preview page numper 35
book preview page numper 36
book preview page numper 37
book preview page numper 38
book preview page numper 39
book preview page numper 40
390 pages