Show Less
Restricted access

Variability in assessor responses to undergraduate essays

An issue for assessment quality in higher education

Sally Roisin O'Hagan

Academic standards in higher education depend on the judgements of individual academics assessing student work; it is in these micro-level practices that the validity and fairness of assessment is constituted. However, the quality of assessments of open-ended tasks like the coursework essay is difficult to ascertain because of the complex and subjective nature of the judgements involved. In view of current concerns about assessment quality and standards, this book is a timely reflection on the practices of academic judgement at university. It explores assessment quality through an empirical study of essay marking in an undergraduate discipline where large class sizes and significant numbers of second language students are common. The study shows that assessors vary in their interpretations of criteria and standards and that this results in inconsistent grading of essays. The book contributes to a growing scholarship of assessment with an evidence-based explanation of why assessors disagree and a discussion of the implications of this for the validity of assessment practices at university.
Show Summary Details
Restricted access

3. Method



This chapter comprises four sections in which are presented i) the research questions that guided this study; ii) a pilot study; iii) a data collection trial; and iv) the main study. The pilot study and data collection trial, carried out prior to the main study, are both reported here because they provide an account of how and why the procedures used in the main study were chosen. The description of the pilot study provides insight into the researcher’s familiarity with the research domain, knowledge which was required for designing the main study; while the reporting of the trial study provides an account of the development of the data collection procedures used in the main study. In the section on the main study, the reader is familiarised with the research site, the participants, the materials and procedures used in the study, and finally, the methods used for data treatment and analysis.

3.1Research questions

As the discussion in Chapter 2 has shown, in the context of the assessment of university coursework essays, there is much yet to be explored concerning the issue of whether, and how, assessor responses vary according to student language background. In this study, the impact of student language background on essay marking is investigated by comparing assessor responses to essays in two language groups: essays by English native-speaking students (NS essays) and non-native speaking students (NNS essays). Three aspects of assessor response are explored: i) the marks awarded by assessors; ii)...

You are not authenticated to view the full text of this chapter or article.

This site requires a subscription or purchase to access the full text of books or journals.

Do you have any questions? Contact us.

Or login to access all content.