Show Less
Restricted access

Variability in assessor responses to undergraduate essays

An issue for assessment quality in higher education

Sally Roisin O'Hagan

Academic standards in higher education depend on the judgements of individual academics assessing student work; it is in these micro-level practices that the validity and fairness of assessment is constituted. However, the quality of assessments of open-ended tasks like the coursework essay is difficult to ascertain because of the complex and subjective nature of the judgements involved. In view of current concerns about assessment quality and standards, this book is a timely reflection on the practices of academic judgement at university. It explores assessment quality through an empirical study of essay marking in an undergraduate discipline where large class sizes and significant numbers of second language students are common. The study shows that assessors vary in their interpretations of criteria and standards and that this results in inconsistent grading of essays. The book contributes to a growing scholarship of assessment with an evidence-based explanation of why assessors disagree and a discussion of the implications of this for the validity of assessment practices at university.
Show Summary Details
Restricted access

4. Results



In explaining the focus of this study, two research questions were asked in the first section of the previous chapter: How do marks awarded to NS and NNS essays vary according to essay language group? and How does assessors’ evaluative verbal commentary on NS and NNS essays vary according to essay language group? This chapter provides answers to these questions in three main sections, each of which compares assessors’ responses to the essays by language group – the first section comparing the marks; the second, the verbal commentary at the level of the major coding categories; and the third, the verbal commentary at the sub-category level. The findings reported in the first of these sections raise a third issue – variability of marks – which is examined in the last section of the chapter in terms of i) the range of marks given to each essay and how this is manifest in the two language groups, and ii) the reliability of assessors’ marks.

4.1Essay marks

During the marking sessions, the ten assessors awarded a score out of 30 to the ten essays, resulting in a total of 100 assessments – 50 for the NS essays, and 50 for the NNS essays. The first part of this section compares the marks for each language group, reporting the average scores and grade distributions. Following this, to examine the extent of within group variation, the average marks are reported for the individual essays in each language group (each essay having...

You are not authenticated to view the full text of this chapter or article.

This site requires a subscription or purchase to access the full text of books or journals.

Do you have any questions? Contact us.

Or login to access all content.