Analyzing and Evaluating Rater-Mediated Assessments. 2nd Revised and Updated Edition
Currency depends on your shipping address
- Frankfurt am Main, Berlin, Bern, Bruxelles, New York, Oxford, Wien, 2015. 241 pp.
- About the author
- About the book
- This eBook can be cited
- Preface to the First Edition
- Preface to the Second Edition
- 1. Introduction
- 1.1 Facets of measurement
- 1.2 Purpose and plan of the book
- 2. Rasch Measurement: The Basics
- 2.1 Elements of Rasch measurement
- 2.1.1 The dichotomous Rasch model
- 2.1.2 Polytomous Rasch models
- 2.2 Rasch modeling of many-facet data
- 2.2.1 Putting the facets together
- 2.2.2 The sample data: Essay ratings
- 2.2.3 Rasch modeling of essay rating data
- 3. Rater-Mediated Assessment: Meeting the Challenge
- 3.1 Rater variability
- 3.2 Interrater reliability
- 3.2.1 The standard approach
- 3.2.2 Consensus and consistency
- 3.2.3 Limitations of the standard approach
- 3.3 A conceptual–psychometric framework
- 3.3.1 Proximal and distal facets
- 3.3.2 A measurement approach
- 4. Many-Facet Rasch Analysis: A First Look
- 4.1 Preparing for a many-facet Rasch analysis
- 4.2 Measures at a glance: The Wright map
- 4.3 Defining separation statistics
- 4.4 Applying separation statistics
- 4.5 Global model fit
- 5. A Closer Look at the Rater Facet: Telling Fact from Fiction
- 5.1 Rater measurement results
- 5.1.1 Estimates of rater severity
- 5.1.2 Rater fit statistics
- 5.1.3 Observed and fair rater averages
- 5.2 Studying central tendency and halo effects
- 5.2.1 Central tendency
- 5.2.2 Halo
- 5.3 Raters as independent experts
- 5.4 Interrater reliability again: Resolving the paradox
- 6. Analyzing the Examinee Facet: From Ratings to Fair Scores
- 6.1 Examinee measurement results
- 6.2 Examinee fit statistics
- 6.3 Examinee score adjustment
- 6.4 Criterion-specific score adjustment
- 7. Criteria and Scale Categories: Use and Functioning
- 7.1 Criterion measurement results
- 7.2 Rating scale structure
- 7.3 Rating scale quality
- 8. Advanced Many-Facet Rasch Measurement
- 8.1 Scoring formats
- 8.2 Dimensionality
- 8.3 Partial credit and hybrid models
- 8.4 Modeling facet interactions
- 8.4.1 Exploratory interaction analysis
- 8.4.2 Confirmatory interaction analysis
- 8.5 Summary of model variants
- 9. Special Issues
- 9.1 Rating designs
- 9.2 Rater feedback
- 9.3 Standard setting
- 9.4 Generalizability theory (G-theory)
- 9.5 MFRM software and extensions
- 10. Summary and Conclusions
- 10.1 Major steps and procedures
- 10.2 MFRM across the disciplines
- 10.3 Measurement and validation
- 10.4 MFRM and the study of rater cognition
- 10.5 Concluding remarks
- Author Index
- Subject Index
Adams, R. J., Wilson, M., & Wang, W-C. (1997). The multidimensional random coefficients multinomial logit model. Applied Psychological Measurement, 21, 1–23.
Adams, R. J., & Wu, M. L. (2007). The mixed-coefficients multinomial logit model: A generalized form of the Rasch model. In M. von Davier & C. H. Carstensen (Eds.), Multivariate and mixture distribution Rasch models: Extensions and applications (pp. 57–75). New York, NY: Springer.
Adams, R., Wu, M., & Wilson, M. (2012). ConQuest (Version 3.0) [Computer software]. Camberwell, Australia: ACER Press.
Akiyama, T. (2001). The application of G-theory and IRT in the analysis of data from speaking tests administered in a classroom context. Melbourne Papers in Language Testing, 10, 1–21.
Alderson, J. C., & Banerjee, J. (2001). Language testing and assessment (Part 1). Language Teaching, 34, 213–236.
Alderson, J. C., & Banerjee, J. (2002). Language testing and assessment (Part 2). Language Teaching, 35, 79–113.
American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (2014). Standards for educational and psychological testing. Washington, DC: American Educational Research Association.
Andersen, E. B. (1982). Georg Rasch (1901–1980). Psychometrika, 47, 375–376.
Andersen, E. B. (2005). Rating scale model. Encyclopedia of Social Measurement, 3, 307–315.
You are not authenticated to view the full text of this chapter or article.
This site requires a subscription or purchase to access the full text of books or journals.
Do you have any questions? Contact us.Or login to access all content.