The 1970s and 1980s were times when communication behavior was a primary interest of many communication scholars. The aim of this book is to reignite some interest in and passion about how human communication behavior should be studied. It presents the best advice, techniques, cautions, and controversies from the 1970s and 1980s and then updates them. Several chapters also introduce statistical methods and procedures to allow readers to analyze behavioral data.
This book is a useful resource for communication scholars and graduate students to guide their study of communication behavior.
7 Reliability for Quantitative Observational Data
| 145 →
Reliability for Quantitative Observational Data
The purpose of this chapter is to introduce several approaches to assessing reliability of observational data that are at least interval level in their measurement. Such data would include (1) frequency counts obtained from the observe and record approach, (2) durational measures obtained via timed-event sequential data (Bakeman et al., 2009), or (3) ratings obtained from a carefully constructed set of rating scales. Ratings scales per se have not been and will not be discussed in this book, although they are certainly a viable approach to behavioral observation. Many texts on social science research methods contain a section on rating scales, so that presentation will not be repeated here.
The reason for a separate chapter on the reliability of interval level or higher data (henceforth called quantitative data because they represent “quantities”) is that the reliability indices from the previous chapter are not applicable to such data. Scott’s π and Cohen’s κ are designed specifically to assess the reliability of nominal or ordinal level data that are obtained from unitizing and categorizing a stream of communication behavior. Moreover, because both ratings and observe and record data do not involve unitizing, there is no need for a unitizing coefficient of reliability such as Guetzkow’s U.