Show Less
Restricted access

Testing ESL Sociopragmatics

Development and Validation of a Web-based Test Battery


Carsten Roever, Catriona Fraser and Catherine Elder

Testing of second language pragmatics has grown as a research area but still suffers from a tension between construct coverage and practicality. In this book, the authors describe the development and validation of a web-based test of second language pragmatics for learners of English. The test has a sociopragmatic orientation and strives for a broad coverage of the construct by assessing learners’ metapragmatic judgments as well as their ability to co-construct discourse. To ensure practicality, the test is delivered online and is scored partially automatically and partially by human raters. We used the argument-based approach to validation, which showed that the test can support low-stakes decisions about learners’ knowledge of sociopragmatics in English.
Show Summary Details
Restricted access

6 This study

← 70 | 71 →6 This study


Our main goal in this study was to develop a test of second language sociopragmatic knowledge for ESL learners in the Australian context. The test was to have broad construct coverage of sociopragmatics and be highly practical in that it can be administered over the internet, take less than an hour and not include a live interlocutor. We designed the test with the goal to enable low-stakes decisions about learners’ sociopragmatic competence, such as pedagogical placement, planning of self-access study, formative feedback and curriculum planning. For the design and validation of the test, we adopted Kane’s (2006) argument-based validity approach with Chapelle’s (2008) modifications. In order to collect backings for our validity argument, we designed a range of empirical studies that would allow us to evaluate to what degree decisions could be justifiably based on our test.

We focus on learners’ knowledge rather than performance since performance would invariably require spoken interaction and establishment of situational context, which would be extremely resource intensive. Practicality was a central concern of our project since we are keenly aware that an impractical test is not likely to be used. At the same time, the decision to focus on knowledge rather than performance limits the range of conclusions that can be drawn from our scores about test takers’ actual language use in real-world settings.

The motivation for designing the test for the Australian context was due to several reasons. Our project had received funding from the Australian Research Council’s Discovery...

You are not authenticated to view the full text of this chapter or article.

This site requires a subscription or purchase to access the full text of books or journals.

Do you have any questions? Contact us.

Or login to access all content.