Show Less
Restricted access

Validating Analytic Rating Scales

A Multi-Method Approach to Scaling Descriptors for Assessing Academic Speaking


Armin Berger

This book presents a unique inter-university scale development project, with a focus on the validation of two new rating scales for the assessment of academic presentations and interactions. The use of rating scales for performance assessment has increased considerably in educational contexts, but the empirical research to investigate the effectiveness of such scales is scarce. The author reports on a multi-method study designed to scale the level descriptors on the basis of expert judgments and performance data. The salient characteristics of the scale levels offer a specification of academic speaking, adding concrete details to the reference levels of the Common European Framework. The findings suggest that validation procedures should be mapped onto theoretical models of performance assessment.
Show Summary Details
Restricted access

1 Introduction


1  Introduction

Although rating scales as operationalisations of speaking test constructs are extensively used in oral performance assessment, they often lack empirical validation. One of the main concerns is that rating scales which were developed intuitively based on expert judgements fail to represent the progression of speaking proficiency in an appropriate way (Brindley 1998; Hulstijn 2007; Kramsch 1986; Lantolf & Frawley 1985; Savignon 1985). It is far from clear whether or not, and if, to what extent, rating scales in fact describe an implicational continuum of increasing language proficiency that corresponds to real language use. Accordingly, there is need for extensive research in the field of educational language testing to show that the speaking constructs and their operationalisations in rating scales are related to the reality of language use (Kaftandjieva & Takala 2003). The study reported in this book sought to do exactly that. It investigated the operationalisation of a speaking construct in two analytic rating scales developed by the Austrian English Language Teaching and Testing (ELTT) initiative, aiming to ascertain whether the level descriptors actually represent an incremental pattern of increasing speaking proficiency. The introductory chapter of this book first describes the background to the research. Then it goes on to state the problem and purpose of the study. Finally, it lists the major research questions and outlines the structure of the book.

1.1  Background to the study

The research presented here grew out of a unique inter-university construct definition project initiated by...

You are not authenticated to view the full text of this chapter or article.

This site requires a subscription or purchase to access the full text of books or journals.

Do you have any questions? Contact us.

Or login to access all content.