Developments in Glocal Language Testing
The Case of the Greek National Foreign Language Exam System
Summary
The chapters included in this volume highlight many of the innovative features of the KPG exam suite and focus on various issues relating to the reliability and validity of glocal language tests. These include the effect of text and reader variables on reading comprehension, different world representations and ideologies in reading texts, interlocutor performance variability, source text regulated written mediation performance, writing and listening task difficulty, and the effect of task and assessment variables and corpus-based research of text grammar on KPG candidates’ scripts – all of which have important implications for global language testing and practices.
Excerpt
Table Of Contents
- Cover
- Title
- Copyright
- About the author(s)/editor(s)
- About the book
- This eBook can be cited
- Contents
- List of Figures
- List of Tables
- List of Abbreviations
- Foreword (Elana Shohamy)
- Preface
- Introduction Issues and challenges in glocal language testing: The case of the Greek State Certificate in Language Proficiency (Evdokia Karavas / Bessie Mitsikopoulou)
- The KPG system: A glocal multilingual examination suite
- Additional innovatory features of the KPG exam system
- Researching the KPG English exam
- Conclusion
- Bibliography
- Appendix: Introducing the KPG as a national project
- Part I Understanding and producing oral texts
- 1 The sociosemiotic language view underlying the design of the KPG exams (Bessie Mitsikopoulou / Evdokia Karavas / Christina Lykou)
- A sociosemiotic view of language
- The impact of the sociosemiotic approach on test tasks
- A genre-based approach to writing
- Conclusion
- Bibliography
- 2 Oral text difficulty in foreign language listening comprehension testing (Elisabeth Apostolou)
- Introduction
- Background to the study
- Linguistic variables affecting oral text difficulty: Syntactic complexity and lexical difficulty
- Paralinguistic variables affecting oral text difficulty: The role of speech rate and accent
- Cognitive variables: The role of background knowledge
- Research methodology
- Quantitative data: Candidates’ questionnaires
- Qualitative data: Interviews and verbal reports
- Presentation of findings and discussion
- Candidate questionnaire data
- Learners’ verbal reports
- Discussion and conclusion
- Bibliography
- 3 The impact of oral examiners’ conduct on candidates’ performance in the KPG exam (Xenia Delieza)
- Introduction: Oral language assessment and the KPG oral exam in English
- Literature review
- The examiner variable in oral language proficiency testing
- Background of the study
- The KPG speaking test
- The Oral Examiner Training Programme
- The Observation Project
- Purpose of the study and methodology
- Data collection and analysis
- Findings: Types of examiner involvement: Quantity and quality
- Results from observation
- The B2 level oral exam
- Results from simulated tests
- Comparisons and conclusions
- Discussion and implications
- Conclusion
- Bibliography
- Appendix I
- Appendix II
- Part II Understanding written texts
- 4 Investigating ideologies in texts used in global and ‘glocal’ language testing (Amalia Balourdi)
- Introduction
- Literature review
- Ideology and discourse
- Transitivity analysis in SFL
- Purpose of the study and methodology
- Quantitative analysis
- Qualitative analysis: The construal of three different realities around ‘giving’ and ‘taking’
- Conclusion
- Bibliography
- 5 Visual matters in language testing: Exploring intersemiotic meaning making in reading comprehension source texts (Styliani Karatza)
- Introduction
- SFL and image/text relations in terms of content: Intersemiosis
- Investigating intersemiotic visual–verbal relations through Systemic Functional Multimodal Discourse Analysis (SF-MDA)
- Findings about visual-verbal relations of KPG reading comprehension source texts
- Analysis of gist items
- Discussion about ideationally oriented intersemiotic literacy requirements and implications
- Conclusion
- Bibliography
- 6 Test-takers’ strategies in responding to KPG English language reading comprehension tasks: Evidence from retrospective think-aloud protocols (Trisevgeni Liontou)
- Introduction
- Literature review
- Research methodology
- Findings
- Item type classification
- Strategy use in main idea items
- Strategy use in factual information items
- Discussion
- Conclusion
- Bibliography
- Acknowledgements
- Appendix
- Part III Producing written texts
- 7 A comparative study of cohesion in L2 candidates’ texts (Virginia-Maria Blani)
- Introduction
- Literature review in L1 and L2 research on cohesion
- Research context, data and methodology
- Corpus
- Methodology
- The adopted model for the cohesion analysis
- Data analysis procedure
- Presentation of the results and discussion
- Grammatical cohesion
- Conjunctions
- Reference
- Lexical Cohesion
- Conclusion and implications
- Bibliography
- 8 The effect of source text on read-to-write task difficulty (Vasso Oikonomidou)
- The aim and context of the study
- Read-to-write tasks in foreign language exams
- Research on read-to-write task difficulty
- Source text related factors affecting read-to-write task difficulty
- Research methods: Two types of analysis
- Aim and research questions
- The data: B1 level source texts and task scores
- Methods of data analysis
- Research findings
- Source text analysis
- Scoring analysis
- Discussion of findings
- Conclusions and implications
- Bibliography
- Appendix
- 9 Assessing cross-language mediation in the Greek national multilingual exam suite: Test-takers’ language performance and strategy use (Maria Stathopoulou)
- Introduction: The context of the study
- Cross-language mediation in foreign language didactics
- Mediation: A neglected area?
- Towards an understanding of the notion of cross-language mediation
- Mediation tasks and mediation strategies
- Aim of the study
- Data collection methods
- Presentation and discussion of results
- The popularity and frequency of mediation strategies in successful and less successful scripts
- Conclusion
- Bibliography
- Appendix I
- Appendix II
- Part IV Electronic developments
- 10 Documenting language proficiency levels using learner data from the KPG examinations suite (Voula Gotsoulia)
- Enriching language proficiency descriptions
- A multilingual database with descriptions of levelled language competences
- Language performance data
- The representation of interrelated language competences
- A lexical semantic model for corpus annotation
- Illustrative corpus analyses for English
- Conclusion
- Bibliography
- 11 KPG e-school: The diffusion and implementation of an educational innovation (Bessie Mitsikopoulou / Evdokia Karavas / Smaragda Papadopoulou)
- Introduction: Setting the context
- Designing the KPG e-school
- Strategy apps in the students’ electronic class
- Purpose
- Structure
- Design and development process
- Specifications, principles and features
- E-books and activity database in the teacher’s electronic class
- Purpose
- Structure
- Design and development process of the activity database
- Specifications, principles and features
- The parent’s electronic class
- Purpose
- Structure
- Conclusion
- Bibliography
- Conclusion (G)Local alternatives in testing (Evdokia Karavas / Bessie Mitsikopoulou)
- Localised language testing
- Glocal language testing
- Bibliography
- Appendix
- The structure and content of the KPG exams
- A level exam
- B level exam
- C level exam
- Notes on Contributors
- Index
- Series index
Figure 5.1: TEXT: Workaholic
Figure 5.2: TEXT: Politeness
Figure 5.3: TEXT: satellites
Figure 5.4: TEXT: women’s history
Figure 5.5: TEXT: Are we alone
Figure 5.6: TEXT: Pay attention
Figure 11.1: The reading strategy app for KPG exam in English (B level)
Table 2.1: Questionnaire survey sample
Table 2.2: Subjects’ profile
Table 2.3: Questionnaires’ high difficulty rates
Table 3.1: The content and structure of the KPG B2 and C1 speaking tests
Table 3.2: Frequencies of examiner intervention per test task in the B2 speaking test in May 2007 and November 2007 observation phases
Table 3.3: Frequencies of examiner intervention per test task in the C1 level speaking tests in May 2007 and November 2007 observation phases
Table 3.4: Number of instances of involvement in the two levels per Activity
Table 3.5: Types of involvement as identified in the simulated speaking tests and percentages for each Activity
Table 3.6: List of acceptable versus non-acceptable intervention
Table 3.7: Percentages of different types of changes of rubrics in the B2 and C1 oral test in May
Table 3.8: Percentages of ‘other’ types of changes of rubrics (in the B2 and C1 oral test in May 2007) classified as acceptable and non-acceptable
Table 3.9: Percentages of different types of interruption of candidates in the B2 and C1 oral test in May
Table 3.10: Percentages of ‘other’ types of interruption of candidates (in the B2 and C1 oral test in May 2007) classified as acceptable and non-acceptable
Table 4.1: Types and sub-types of material processes across the three sub-corpora ← xi | xii →
Table 4.2: Frequencies of human actants across the three sub-corpora
Table 4.3: Frequencies of non–human actants across the three sub-corpora
Table 5.1: Instances of visual–verbal relations
Table 5.2: Visual–verbal relations: Grouping of data instantiations
Table 5.3: Types of Gist Items (TGIs)
Table 6.1: Sample of KPG reading comprehension test item types
Table 6.2: Strategies for correct responses to main idea items
Table 6.3: Strategies for incorrect responses to main idea items
Table 6.4: Strategies for correct responses to factual information items
Table 6.5: Strategies for incorrect responses to factual information items
Table 6.6: Reading strategies coding system (adapted from Cohen and Upton 2007)
Table 6.7: Test-taking strategies coding system (adapted from Cohen and Upton 2007)
Table 7.1: Grammatical cohesion analysis results
Table 7.2: Types of conjunction per script category and statistical significance chi-square test results
Table 7.3: Chi-square significance test results for adversative conjunctions
Table 7.4: Chi-square significance tests results for causal conjunctions
Table 7.5: Types of reference per script category (raw frequency per type) and chi-square test significant difference results
Table 7.6: Raw and relative frequencies of lexical cohesion types
Table 7.7: Chi-square test results for lexical cohesion sub-types
Table 8.1: Text related factors affecting task difficulty
Table 8.2: Source text related factors in this investigation ← xii | xiii →
Table 8.3: Source text complexity indices
Table 8.4: Readability indices of source texts
Table 8.5: Syntactic complexity indices of source texts
Table 8.6: Lexical complexity indices of source texts
Table 8.7: Test taker performance on read-to-write tasks
Table 8.8: Task analysis
Table 9.1: Types of mediation strategies
Table 9.2: Degree of popularity of mediation strategies in fully satisfactory and moderately satisfactory scripts
Table 9.3: Type A and Type B mediation strategies in fully and moderately satisfactory scripts across levels: Frequencies
Table 9.4: Mediation strategy use in fully satisfactory and moderately satisfactory scripts across levels: Frequency
Table 9.5: Mean number of words used to realise mediation strategies
Table 9.6: Mediation strategies spotted in the corpus (Stathopoulou 2015)
Table 9.7: Mediation strategy use in fully satisfactory and moderately satisfactory scripts across levels: Frequency
Table 10.1: Language proficiency descriptors linked with lexico-grammatical data
Table 10.2: The Judgment frame
Table 10.3: Output of the Semafor parser
Table 10.4: Lexical coverage of FrameNet (FN) in sample KPG sub-corpus
Table 10.5: Discrete frame occurrences in sample sub-corpus and sentences per script
Table 10.6: Levelled, contextual lexical use in Judgment
Table 10.7: Levelled, contextual use of ‘admire’ in Judgment
Table 11.1: Contents and modalities used in Modules 1 and 4
Adv Adverbial
ANOVA Analysis of Variance
CALT Computer Adaptive Language Testing
CDA Critical Discourse Analysis
CEFR Common European Framework of Reference for Languages
CESOL Cambridge ESOL exams
CLD Curriculum Language Database
CLC Cambridge Learner Corpus
CMS Content Management System
EFL English as a Foreign Language
ELIUM Michigan Language Exams
ELL Early Language Learning
EPP English Profile Programme
ESL English as a Second Language
EU European Union
FAQs Frequently Asked Questions
FL Foreign Language
IELTS International English Language Testing System
IFLC Integrated Foreign Languages Curriculum
IMS Inventory of Mediation Strategies
KPG Kratiko Pistopiitiko Glossomathias
L1 First Language
L2 Second Language
LPI Language Proficiency Interview
MDA Multimodal Discourse Analysis
MMA Multimodal Analysis Image
NP Noun Phrase
OPI Oral Proficiency Interview
PP Prepositional Phrase
RCeL Research Centre for Language Teaching, Testing and Assessment ← xv | xvi →
SFL Systemic Functional Linguistics
TGI Test Gist Item
TSRP Test Taking Strategies Research Project
TWE Test of Written English
V Verb
WCMS Web Content Management System
It is a special honour to be asked by Evdokia Karavas and Bessie Mitsikopoulou, the editors of this volume, to write a foreword to this special book in honour of Professor Bessie Dendrinos. The book, entitled Developments in Glocal Language Testing: The Case of the Greek National Foreign Language Proficiency Exam, represents the state of the art in language testing, since it integrates language testing with multiple societal and educational perspectives, that is, the political, the social, language policy, linguistics and psychometrics. This integration is manifested through the case of the Greek National Foreign Language Proficiency Exam, an examination battery that Professor Dendrinos devoted her work to in the past two decades. Thus, the findings reported in the chapters of this book are derived from research and development within the Greek testing system. Yet, the implications of these findings go far beyond the Greek case as they are relevant to the language testing field at large in many worldwide systems. Hence, the word ‘glocal’ in the title implies both the local and the global. The other unique dimension of the chapters is that the research described in the volume was conducted with, and/or under the direction of Professor Dendrinos, with her creative and critical thinking.
I would like to acknowledge and recognise the special contribution that Professor Dendrinos made to the Applied Linguistics field and to my own thinking and academic growth. Bessie is a scholar who affected many researchers and had an influence on their critical views, seeking language justice in relation to English and other languages.
In the mid-1990s, I attended a lecture delivered by Professor Dendrinos at one of the GURT conferences held at Georgetown University in Washington DC, organised by the late Professor James Alatis. I had the privilege of listening to Professor Dendrinos addressing issues which at the time were very new to me and required great courage to discuss. This was an ← xvii | xviii → era when the number of people learning English was growing exponentially, especially in the third-world countries where governments and Ministries of Education were introducing English in school systems, in the belief that knowledge of English would improve the economy and status of their counties. It was during that lecture that Professor Dendrinos delivered the message about the power of the English language and the inequalities it produced, especially in relation to the western world. Bessie demonstrated this in a number of ways, but the one I recall best was how the contents included in the English textbooks gave false ideas to the people of the third world that learning the English language would make them successful in life. She argued that English textbooks portray those who know the English language as wealthy, rich, smart and having endless opportunities and possibilities. Quite often, according to Dendrinos, the English language textbooks equated speakers of English with the ‘haves’, leading the learners in the third world to believe that if they were proficient in the English language, they would be able to attain the same status and wealth.
Details
- Pages
- XXIV, 364
- Publication Year
- 2019
- ISBN (PDF)
- 9781787076921
- ISBN (ePUB)
- 9781787076938
- ISBN (MOBI)
- 9781787076945
- ISBN (Softcover)
- 9783034322416
- DOI
- 10.3726/b11390
- Language
- English
- Publication date
- 2019 (January)
- Keywords
- research in language testing glocal testing Foreign language proficiency examinations
- Published
- Oxford, Bern, Berlin, Bruxelles, New York, Wien, 2019. XXIV, 364 pp., 7 fig. b/w, 56 tables
- Product Safety
- Peter Lang Group AG