Show Less

Task Equivalence in Speaking Tests

Investigating the Difficulty of Two Spoken Narrative Tasks


Chihiro Inoue

This book addresses the issue of task equivalence, which is of fundamental importance in the areas of language testing and task-based research, where task equivalence is a prerequisite. The main study examines the two ‘seemingly-equivalent’ picture-based spoken narrative tasks, using a multi-method approach combining quantitative and qualitative methodologies with MFRM analysis of the ratings, the analysis of linguistic performances by Japanese candidates and native speakers of English (NS), expert judgements of the task characteristics, and perceptions of the candidates and NS. The results reveal a complex picture with a number of variables involved in ensuring task equivalence, raising relevant issues regarding the theories of task complexity and the commonly-used linguistic variables for examining learner spoken language. This book has important implications for the possible measures that can be taken to avoid selecting non-equivalent tasks for research and teaching.


Show Summary Details
Restricted access

4. Methodology 123


4. Methodology 4.1 Introduction This chapter explains the methodology used in the main study of this book. First the research questions are presented, then there follows a description of data collection from the Japanese candidates, Japanese teachers and native speakers of English. Then, the procedures for ob- taining ratings data, including rater training, are explained. Finally, the methods of analysis are introduced, corresponding to each of the eight research questions. In light of the literature reviewed in Chapter 2 and the results of the pilot studies described in Chapter 3, eight research questions were established, as listed below. The research questions can be broadly divided into four parts, investigating the difficulty of the two spoken narrative tasks by MFRM analysis (RQ1); the perceptions of the can- didates, expert judgements by Japanese teachers and perceptions of the native speakers of English (RQ2); the linguistic performances of the candidates (RQ3); and the validity of the linguistic variables (RQ4). RQ1. Is the difficulty of the two spoken narrative tasks (taken from Hill (1960)) the same according to MFRM analysis? RQ2-1. Are the candidates’ perceptions of the two spoken narrative tasks the same? RQ2-2. Are the candidates’ perceptions of the two spoken narrative tasks the same at different levels of proficiency? RQ2-3. Do experts judge the two spoken narrative tasks to be equivalent for the candidates in terms of the relevant task complexity factors? RQ2-4. Do English native speakers perceive the two spoken narra- tive tasks to be equally difficult? RQ3-1. Are the performances...

You are not authenticated to view the full text of this chapter or article.

This site requires a subscription or purchase to access the full text of books or journals.

Do you have any questions? Contact us.

Or login to access all content.