Date of Award

August 2024

Degree Type

Dissertation

Degree Name

Doctor of Philosophy

Department

Educational Psychology

First Advisor

Razia Azen

Committee Members

Bo Zhang, Kyongboon Kwon, Luciana Cançado

Keywords

Dominance Analysis, Multilevel Models

Abstract

Background: Dominance Analysis (DA) was originally proposed to determine the relative importance of predictor variables in OLS regression models by comparing the change in model fit (i.e., R2) resulting from adding each predictor to each possible subset model (Azen & Budescu, 2003; Azen, 2013; Budescu, 1993). Although various educational studies show that DA can provide useful information in research, the DA procedure has not been studied extensively with Multilevel Linear Models (MLMs), which are commonly used to analyze nested data structures.

Purpose: This study aimed to identify appropriate multilevel measures of fit for the DA procedure in various MLMs, and determine which measures produce the most accurate relative importance results under different multilevel model conditions. The current study aimed to examine how various factors that can affect the DA performance in order to extend the DA application to MLMs determine the relative importance of predictors. The study evaluated the DA performance in diverse multilevel contexts using the three multilevel ?2 measures (i.e., ???_?, ??_?? and ???_???) which are all based on the full partitioning of the total outcome variance (Johnson, 2014; Nakagawa & Schielzeth, 2013; Rights & Sterba, 2019).

Methods: A Monte Carlo simulation study was conducted to compare and evaluate DA results across several simulation conditions. Conditions included various sample sizes and predictor effects, the way in which the level-1 predictors are centered (i.e., grand- or cluster-mean centering), the multilevel measure of fit used to determine the change in model fit (e.g., the variance component explained, as discussed by Rights and Sterba, 2019), and the type of MLM used (e.g., its random effects). The DA rank ordering of predictors was evaluated using the Kendall rank correlation (Kendall, 1955). Simple random sampling and multilevel bootstrapping were used to evaluate inference for the quantitative general dominance measure (e.g., bias, RMSE, coverage of 95% CIs, Type I error and power rates) and made recommendations for applied research.

Results: The results showed that all measures of fit investigated in this study (i.e., ???_?, ??_??, and ???_???) were appropriate for obtaining an accurate rank ordering of the relative importance of level-1 predictors using DA. The ???_? or ??_?? measures were generally appropriate and can be recommended for DA of level-2 predictors, though relatively large sample sizes may be required for accurate results. The ???_??? measure was also appropriate to determine the relative importance of level-2 predictors with large sample sizes and large dominance relationship effect sizes, but only with the GMC method. It is not recommended that the ???_??? measure be used with the CMC method to determine the predictor importance of the level-2 predictors because it often resulted in undesired levels of negative additional contributions, standardized bias, coverage rates of 95% confidence interval, and type I error rates.

Conclusions and Implication: The current study found that the choice of measure of fitis important when using DA to determine the relative importance of predictors in MLM studies and recommends that researchers carefully consider which explained variance components are meaningful and enhance the accuracy of DA results under their defined multilevel model type and mean centering method. This study contributed useful information for applying DA in MLM studies and provided detailed recommendations and demonstrations of how to apply the procedure using empirical datasets.

Download

Share

COinS