Show Less

Task Equivalence in Speaking Tests

Investigating the Difficulty of Two Spoken Narrative Tasks


Chihiro Inoue

This book addresses the issue of task equivalence, which is of fundamental importance in the areas of language testing and task-based research, where task equivalence is a prerequisite. The main study examines the two ‘seemingly-equivalent’ picture-based spoken narrative tasks, using a multi-method approach combining quantitative and qualitative methodologies with MFRM analysis of the ratings, the analysis of linguistic performances by Japanese candidates and native speakers of English (NS), expert judgements of the task characteristics, and perceptions of the candidates and NS. The results reveal a complex picture with a number of variables involved in ensuring task equivalence, raising relevant issues regarding the theories of task complexity and the commonly-used linguistic variables for examining learner spoken language. This book has important implications for the possible measures that can be taken to avoid selecting non-equivalent tasks for research and teaching.


Show Summary Details
Restricted access

5. Results 149


5. Results 5.1 Introduction This chapter reports the results of the analyses for the research ques- tions and brief interpretations of them. Each section in this chapter corresponds to a research question and starts by describing the data and their preliminary analysis (e.g. order effect) before presenting the main results. The results for RQ1 comprise the MFRM analysis of the difficulty of the two spoken narrative tasks, based on the overall CEFR ratings, as well as the ratings in the five rating categories. The following section describes the results of t-tests for candidate percep- tions of the tasks, both for the whole candidate population (RQ2-1) and for different levels of proficiency based on the MFRM analysis of candidate ability (RQ2-2). In addition, following on from suggestions made in Pilot Study 5, RQ2-3 explores the expert judgements by Japanese teachers on the two tasks, regarding task complexity factors, based on the Checklist of Difficulty by Weir and Wu (2006). Two Japanese teachers of English at TUFS, who were involved in teaching the majority of the candidates, completed the questionnaire. More- over, the results for RQ2-4 present the content of a brief interview about the perceived difficulty of the tasks with the eleven native Eng- lish speakers who performed them both. RQ3 (3-1 and 3-2) describes the linguistic performances for the two tasks by the candidates, again for the whole sample and for different levels of proficiency. NS base- line data are also presented. Finally, RQ4 aims to conduct a validation study...

You are not authenticated to view the full text of this chapter or article.

This site requires a subscription or purchase to access the full text of books or journals.

Do you have any questions? Contact us.

Or login to access all content.