Exploring the relationship between interrater correlations and validity of peer ratings

Ronen Kasten, Baruch Nevo

Research output: Contribution to journalArticlepeer-review

Abstract

A field study was conducted to investigate the relationship between interrater correlations and validity estimates of peer ratings. Validity coefficients and interrater correlations were calculated for 281 work units in a large law enforcement organization in Israel. The main result was a weak positive linear relationship between these two variables. Furthermore, in some of the analyses conducted, a nonlinear quadratic component in the relationship between these measures was evident. Validity was low only when interrater correlation was very low (r =.4 and less). Above this level, validity was stable and almost did not change as interrater correlation increased. This finding, together with other studies (Borman, 1975; Buckner, 1959; Freeberg, 1969; Weekley Gier, 1989), cast doubt on the assertion that interrater correlation in the field of performance rating is a proper measurement of reliability.

Original languageEnglish
Pages (from-to)180-197
Number of pages18
JournalHuman Performance
Volume21
Issue number2
DOIs
StatePublished - Apr 2008

ASJC Scopus subject areas

  • Applied Psychology
  • Psychology (all)
  • Organizational Behavior and Human Resource Management

Fingerprint

Dive into the research topics of 'Exploring the relationship between interrater correlations and validity of peer ratings'. Together they form a unique fingerprint.

Cite this