Criticality of predictors in multiple regression

Razia Azen, David V. Budescu, Benjamin Reiser

Research output: Contribution to journalArticlepeer-review

Abstract

A new method is proposed for comparing all predictors in a multiple regression model. This method generates a measure of predictor criticality, which is distinct from and has several advantages over traditional indices of predictor importance. Using the bootstrapping (resampling with replacement) procedure, a large number of samples are obtained from a given data set which contains one response variable and p predictors. For each sample, all 2p - 1 subset regression models are fitted and the best subset model is selected. Thus, the (multinomial) distribution of the probability that each of the 2p - 1 subsets is 'the best' model for the data set is obtained. A predictor's criticality is defined as a function of the probabilities associated with the models that include the predictor. That is, a predictor which is included in a large number of probable models is critical to the identification of the best-fitting regression model and, therefore, to the prediction of the response variable. The procedure can be applied to fixed and random regression models and can use any measure of goodness of fit (e.g., adjusted R2, Cp, AIC) for identifying the best model. Several criticality measures can be defined by using different combinations of the probabilities of the best-fitting models, and asymptotic confidence intervals for each variable's criticality can be derived. The procedure is illustrated with several examples.

Original languageEnglish
Pages (from-to)201-225
Number of pages25
JournalBritish Journal of Mathematical and Statistical Psychology
Volume54
Issue number2
DOIs
StatePublished - Nov 2001

ASJC Scopus subject areas

  • Statistics and Probability
  • Arts and Humanities (miscellaneous)
  • General Psychology

Fingerprint

Dive into the research topics of 'Criticality of predictors in multiple regression'. Together they form a unique fingerprint.

Cite this