Show Less
Restricted access

Introduction to Many-Facet Rasch Measurement

Analyzing and Evaluating Rater-Mediated Assessments- 2 nd Revised and Updated Edition

Series:

Thomas Eckes

Since the early days of performance assessment, human ratings have been subject to various forms of error and bias. Expert raters often come up with different ratings for the very same performance and it seems that assessment outcomes largely depend upon which raters happen to assign the rating. This book provides an introduction to many-facet Rasch measurement (MFRM), a psychometric approach that establishes a coherent framework for drawing reliable, valid, and fair inferences from rater-mediated assessments, thus answering the problem of fallible human ratings. Revised and updated throughout, the Second Edition includes a stronger focus on the Facets computer program, emphasizing the pivotal role that MFRM plays for validating the interpretations and uses of assessment outcomes.
Show Summary Details
Restricted access

10. Summary and Conclusions

Extract

10. Summary and Conclusions

This final chapter provides a look back on critical concepts and issues of the MFRM approach to the analysis and evaluation of rater-mediated assessments. The first section reconsiders major steps and procedures and presents a summary in a flowchart-like diagram. Throughout the book, a single data set drawn from a writing assessment program was used for illustrative purposes. To broaden the perspective and to underscore the versatility of the MFRM approach, the second section briefly discusses MFRM studies in diverse fields of application, ranging from psychological assessment to medical education and occupational behavior. A major theme of the book has been that conducting a MFRM analysis strongly contributes to ensuring the validity and fairness of interpretations and uses of assessment outcomes. In two separate sections, this theme is explored more deeply.

10.1  Major steps and procedures

Chapter 1 introduced the basic three-step measurement approach to rater-mediated assessments: Identifying potentially relevant facets, specifying an appropriate measurement model, and applying this model to the assessment data. In this section, I elaborate on this approach. Building on a sequence of seven more detailed steps, I review relevant concepts, procedures, and model implementations discussed in the previous chapters. Figure 10.1 presents these steps in a diagrammatic form. Within each box representing a given step, a short list of keywords is provided that illustrate concepts, issues, and procedures typically involved in that step. Arrows between the boxes indicate the usual sequence of steps, which may...

You are not authenticated to view the full text of this chapter or article.

This site requires a subscription or purchase to access the full text of books or journals.

Do you have any questions? Contact us.

Or login to access all content.