A field study was conducted to investigate the relationship between interrater correlations and validity estimates of peer ratings. Validity coefficients and interrater correlations were calculated for 281 work units in a large law enforcement organization in Israel. The main result was a weak positive linear relationship between these two variables. Furthermore, in some of the analyses conducted, a nonlinear quadratic component in the relationship between these measures was evident. Validity was low only when interrater correlation was very low (r =.4 and less). Above this level, validity was stable and almost did not change as interrater correlation increased. This finding, together with other studies (Borman, 1975; Buckner, 1959; Freeberg, 1969; Weekley Gier, 1989), cast doubt on the assertion that interrater correlation in the field of performance rating is a proper measurement of reliability.
ASJC Scopus subject areas
- Applied Psychology
- Psychology (all)
- Organizational Behavior and Human Resource Management