Generalized reliability based on distances

Meng Xu; Philip T. Reiss; Ivor Cribben

doi:10.1111/biom.13287

Generalized reliability based on distances

Meng Xu
, Philip T. Reiss
, Ivor Cribben

Department of Statistics

Research output: Contribution to journal › Article › peer-review

Abstract

The intraclass correlation coefficient (ICC) is a classical index of measurement reliability. With the advent of new and complex types of data for which the ICC is not defined, there is a need for new ways to assess reliability. To meet this need, we propose a new distance-based ICC (dbICC), defined in terms of arbitrary distances among observations. We introduce a bias correction to improve the coverage of bootstrap confidence intervals for the dbICC, and demonstrate its efficacy via simulation. We illustrate the proposed method by analyzing the test-retest reliability of brain connectivity matrices derived from a set of repeated functional magnetic resonance imaging scans. The Spearman-Brown formula, which shows how more intensive measurement increases reliability, is extended to encompass the dbICC.

Original language	English
Pages (from-to)	258-270
Number of pages	13
Journal	Biometrics
Volume	77
Issue number	1
DOIs	https://doi.org/10.1111/biom.13287
State	Published - Mar 2021

Bibliographical note

Publisher Copyright:
© 2020 The Authors. Biometrics published by Wiley Periodicals, Inc. on behalf of International Biometric Society.

Keywords

Spearman-Brown formula
functional connectivity
intraclass correlation coefficient
test-retest reliability

ASJC Scopus subject areas

Statistics and Probability
General Biochemistry, Genetics and Molecular Biology
General Immunology and Microbiology
General Agricultural and Biological Sciences
Applied Mathematics

Access to Document

10.1111/biom.13287

Cite this

@article{c99bfe5b433f40a1b14d0d2125e0e6fc,

title = "Generalized reliability based on distances",

abstract = "The intraclass correlation coefficient (ICC) is a classical index of measurement reliability. With the advent of new and complex types of data for which the ICC is not defined, there is a need for new ways to assess reliability. To meet this need, we propose a new distance-based ICC (dbICC), defined in terms of arbitrary distances among observations. We introduce a bias correction to improve the coverage of bootstrap confidence intervals for the dbICC, and demonstrate its efficacy via simulation. We illustrate the proposed method by analyzing the test-retest reliability of brain connectivity matrices derived from a set of repeated functional magnetic resonance imaging scans. The Spearman-Brown formula, which shows how more intensive measurement increases reliability, is extended to encompass the dbICC.",

keywords = "Spearman-Brown formula, functional connectivity, intraclass correlation coefficient, test-retest reliability",

author = "Meng Xu and Reiss, \{Philip T.\} and Ivor Cribben",

note = "Publisher Copyright: {\textcopyright} 2020 The Authors. Biometrics published by Wiley Periodicals, Inc. on behalf of International Biometric Society.",

year = "2021",

month = mar,

doi = "10.1111/biom.13287",

language = "English",

volume = "77",

pages = "258--270",

journal = "Biometrics",

issn = "0006-341X",

publisher = "Oxford University Press",

number = "1",

}

TY - JOUR

T1 - Generalized reliability based on distances

AU - Xu, Meng

AU - Reiss, Philip T.

AU - Cribben, Ivor

PY - 2021/3

Y1 - 2021/3

N2 - The intraclass correlation coefficient (ICC) is a classical index of measurement reliability. With the advent of new and complex types of data for which the ICC is not defined, there is a need for new ways to assess reliability. To meet this need, we propose a new distance-based ICC (dbICC), defined in terms of arbitrary distances among observations. We introduce a bias correction to improve the coverage of bootstrap confidence intervals for the dbICC, and demonstrate its efficacy via simulation. We illustrate the proposed method by analyzing the test-retest reliability of brain connectivity matrices derived from a set of repeated functional magnetic resonance imaging scans. The Spearman-Brown formula, which shows how more intensive measurement increases reliability, is extended to encompass the dbICC.

AB - The intraclass correlation coefficient (ICC) is a classical index of measurement reliability. With the advent of new and complex types of data for which the ICC is not defined, there is a need for new ways to assess reliability. To meet this need, we propose a new distance-based ICC (dbICC), defined in terms of arbitrary distances among observations. We introduce a bias correction to improve the coverage of bootstrap confidence intervals for the dbICC, and demonstrate its efficacy via simulation. We illustrate the proposed method by analyzing the test-retest reliability of brain connectivity matrices derived from a set of repeated functional magnetic resonance imaging scans. The Spearman-Brown formula, which shows how more intensive measurement increases reliability, is extended to encompass the dbICC.

KW - Spearman-Brown formula

KW - functional connectivity

KW - intraclass correlation coefficient

KW - test-retest reliability

UR - https://www.scopus.com/pages/publications/85085125477

U2 - 10.1111/biom.13287

DO - 10.1111/biom.13287

M3 - Article

C2 - 32339252

AN - SCOPUS:85085125477

SN - 0006-341X

VL - 77

SP - 258

EP - 270

JO - Biometrics

JF - Biometrics

IS - 1

ER -

Generalized reliability based on distances

Abstract

Bibliographical note

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this