fit measures for LMS and QML
fit_measures_da.Rmd
Introduction
This vignette demonstrates how to evaluate and compare model fit for latent interaction models estimated via
- The Latent Moderated Structural Equations (LMS) approach.
- The Quasi-Maximum Likelihood (QML) approach.
using the modsem
package (v≥1.0.8). Because standard
Chi-square statistics are not available under LMS/QML, we assess fit
by:
- Examining fit indices for the baseline (no-interaction) model.
- Conducting a likelihood-ratio difference test to compare the baseline and interaction models (Klein & Moosbrugger, 2000; Klein & Múthen, 2007).
If the baseline model shows acceptable fit and adding the latent interaction significantly improves fit, the interaction model can also be deemed well-fitting.
Example
We define a model with three latent variables (X
,
Y
, Z
) and their interaction
(X:Z
):
m1 <- "
# Outer (measurement) model
X =~ x1 + x2 + x3
Y =~ y1 + y2 + y3
Z =~ z1 + z2 + z3
# Inner (structural) model
Y ~ X + Z + X:Z
"
# Estimate the full (H1) model via LMS
est_h1 <- modsem(m1, oneInt, method = "lms")
# Estimate the baseline (H0) model without interaction
est_h0 <- estimate_h0(est_h1, calc.se = FALSE) # std.errors are not needed
Fit measures baseline model
To get fit measures for the baseline model you can use the
fit_modsem_da()
function.
fit_modsem_da(est_h0)
#> $sigma.observed
#> x1 x2 x3 z1 z2 z3 y1 y2 y3
#> x1 1.141
#> x2 0.788 0.797
#> x3 0.900 0.723 0.986
#> z1 0.210 0.161 0.181 1.185
#> z2 0.174 0.130 0.148 0.825 0.828
#> z3 0.182 0.132 0.151 0.897 0.727 0.949
#> y1 0.788 0.641 0.714 0.734 0.611 0.658 2.648
#> y2 0.625 0.512 0.557 0.601 0.500 0.541 1.988 1.740
#> y3 0.711 0.576 0.642 0.658 0.552 0.598 2.238 1.784 2.176
#>
#> $sigma.expected
#> x1 x2 x3 z1 z2 z3 y1 y2 y3
#> x1 1.141
#> x2 0.789 0.797
#> x3 0.898 0.722 0.985
#> z1 0.201 0.162 0.184 1.184
#> z2 0.163 0.131 0.149 0.824 0.828
#> z3 0.177 0.143 0.163 0.897 0.727 0.948
#> y1 0.786 0.632 0.720 0.749 0.608 0.661 2.646
#> y2 0.627 0.504 0.574 0.598 0.485 0.527 1.987 1.739
#> y3 0.706 0.567 0.646 0.673 0.546 0.594 2.236 1.784 2.175
#>
#> $mu.observed
#> ~1
#> x1 1.023
#> x2 1.215
#> x3 0.919
#> z1 1.011
#> z2 1.206
#> z3 0.916
#> y1 1.180
#> y2 1.335
#> y3 1.083
#>
#> $mu.expected
#> ~1
#> x1 1.023
#> x2 1.215
#> x3 0.919
#> z1 1.011
#> z2 1.206
#> z3 0.916
#> y1 1.180
#> y2 1.335
#> y3 1.083
#>
#> $chisq.value
#> [1] 17.5224
#>
#> $chisq.pvalue
#> [1] 0.8255439
#>
#> $chisq.df
#> [1] 24
#>
#> $AIC
#> [1] 35723.75
#>
#> $AICc
#> [1] 35724.69
#>
#> $BIC
#> [1] 35891.78
#>
#> $aBIC
#> [1] 35796.47
#>
#> $RMSEA
#> [1] 0
#>
#> $RMSEA.lower
#> [1] 0
#>
#> $RMSEA.upper
#> [1] 0.01122746
#>
#> $RMSEA.ci.level
#> [1] 0.9
#>
#> $RMSEA.pvalue
#> [1] 1
#>
#> $RMSEA.close.h0
#> [1] 0.05
It can also be used to get fit measures for the full model, but
should be pared with chisq = FALSE
to avoid the Chi-square
test. If it is set to TRUE
it will calculate the Chi-square
test while ignoring the interaction terms in the model.
fit_modsem_da(est_h1, chisq = FALSE)
#> $sigma.observed
#> x1 x2 x3 z1 z2 z3 y1 y2 y3
#> x1 1.141
#> x2 0.788 0.797
#> x3 0.900 0.723 0.986
#> z1 0.210 0.161 0.181 1.185
#> z2 0.174 0.130 0.148 0.825 0.828
#> z3 0.182 0.132 0.151 0.897 0.727 0.949
#> y1 0.788 0.641 0.714 0.734 0.611 0.658 2.648
#> y2 0.625 0.512 0.557 0.601 0.500 0.541 1.988 1.740
#> y3 0.711 0.576 0.642 0.658 0.552 0.598 2.238 1.784 2.176
#>
#> $sigma.expected
#> NULL
#>
#> $mu.observed
#> ~1
#> x1 1.023
#> x2 1.215
#> x3 0.919
#> z1 1.011
#> z2 1.206
#> z3 0.916
#> y1 1.180
#> y2 1.335
#> y3 1.083
#>
#> $mu.expected
#> NULL
#>
#> $chisq.value
#> NULL
#>
#> $chisq.pvalue
#> NULL
#>
#> $chisq.df
#> NULL
#>
#> $AIC
#> [1] 35049.21
#>
#> $AICc
#> [1] 35050.22
#>
#> $BIC
#> [1] 35222.84
#>
#> $aBIC
#> [1] 35124.35
#>
#> $RMSEA
#> NULL
#>
#> $RMSEA.lower
#> NULL
#>
#> $RMSEA.upper
#> NULL
#>
#> $RMSEA.ci.level
#> NULL
#>
#> $RMSEA.pvalue
#> NULL
#>
#> $RMSEA.close.h0
#> NULL
Difference Test of Fit
Compare H0 vs. H1 using a log-likelihood ratio test:
compare_fit(est_h1 = est_h1, est_h0 = est_h0)
#> $D
#> [1] 676.5418
#>
#> $df
#> [1] 1
#>
#> $p
#> [1] 3.775457e-149
#>
#> $diff.loglik
#> [1] 338.2709
A significant p-value indicates the latent interaction term significantly improves model fit.
Inspecting Fit Indices
For convenience, you can also use the modsem_inspect()
function with what = "fit"
to get fit indices for both
models, and comparative fit in one go.
modsem_inspect(est_h1, what = "fit")
#> $fit.h0
#> $fit.h0$sigma.observed
#> x1 x2 x3 z1 z2 z3 y1 y2 y3
#> x1 1.141
#> x2 0.788 0.797
#> x3 0.900 0.723 0.986
#> z1 0.210 0.161 0.181 1.185
#> z2 0.174 0.130 0.148 0.825 0.828
#> z3 0.182 0.132 0.151 0.897 0.727 0.949
#> y1 0.788 0.641 0.714 0.734 0.611 0.658 2.648
#> y2 0.625 0.512 0.557 0.601 0.500 0.541 1.988 1.740
#> y3 0.711 0.576 0.642 0.658 0.552 0.598 2.238 1.784 2.176
#>
#> $fit.h0$sigma.expected
#> x1 x2 x3 z1 z2 z3 y1 y2 y3
#> x1 1.141
#> x2 0.789 0.797
#> x3 0.898 0.722 0.985
#> z1 0.201 0.162 0.184 1.184
#> z2 0.163 0.131 0.149 0.824 0.828
#> z3 0.177 0.143 0.163 0.897 0.727 0.948
#> y1 0.786 0.632 0.720 0.749 0.608 0.661 2.646
#> y2 0.627 0.504 0.574 0.598 0.485 0.527 1.987 1.739
#> y3 0.706 0.567 0.646 0.673 0.546 0.594 2.236 1.784 2.175
#>
#> $fit.h0$mu.observed
#> ~1
#> x1 1.023
#> x2 1.215
#> x3 0.919
#> z1 1.011
#> z2 1.206
#> z3 0.916
#> y1 1.180
#> y2 1.335
#> y3 1.083
#>
#> $fit.h0$mu.expected
#> ~1
#> x1 1.023
#> x2 1.215
#> x3 0.919
#> z1 1.011
#> z2 1.206
#> z3 0.916
#> y1 1.180
#> y2 1.335
#> y3 1.083
#>
#> $fit.h0$chisq.value
#> [1] 17.5224
#>
#> $fit.h0$chisq.pvalue
#> [1] 0.8255439
#>
#> $fit.h0$chisq.df
#> [1] 24
#>
#> $fit.h0$AIC
#> [1] 35723.75
#>
#> $fit.h0$AICc
#> [1] 35724.69
#>
#> $fit.h0$BIC
#> [1] 35891.78
#>
#> $fit.h0$aBIC
#> [1] 35796.47
#>
#> $fit.h0$RMSEA
#> [1] 0
#>
#> $fit.h0$RMSEA.lower
#> [1] 0
#>
#> $fit.h0$RMSEA.upper
#> [1] 0.01122746
#>
#> $fit.h0$RMSEA.ci.level
#> [1] 0.9
#>
#> $fit.h0$RMSEA.pvalue
#> [1] 1
#>
#> $fit.h0$RMSEA.close.h0
#> [1] 0.05
#>
#>
#> $fit.h1
#> $fit.h1$sigma.observed
#> x1 x2 x3 z1 z2 z3 y1 y2 y3
#> x1 1.141
#> x2 0.788 0.797
#> x3 0.900 0.723 0.986
#> z1 0.210 0.161 0.181 1.185
#> z2 0.174 0.130 0.148 0.825 0.828
#> z3 0.182 0.132 0.151 0.897 0.727 0.949
#> y1 0.788 0.641 0.714 0.734 0.611 0.658 2.648
#> y2 0.625 0.512 0.557 0.601 0.500 0.541 1.988 1.740
#> y3 0.711 0.576 0.642 0.658 0.552 0.598 2.238 1.784 2.176
#>
#> $fit.h1$sigma.expected
#> NULL
#>
#> $fit.h1$mu.observed
#> ~1
#> x1 1.023
#> x2 1.215
#> x3 0.919
#> z1 1.011
#> z2 1.206
#> z3 0.916
#> y1 1.180
#> y2 1.335
#> y3 1.083
#>
#> $fit.h1$mu.expected
#> NULL
#>
#> $fit.h1$chisq.value
#> NULL
#>
#> $fit.h1$chisq.pvalue
#> NULL
#>
#> $fit.h1$chisq.df
#> NULL
#>
#> $fit.h1$AIC
#> [1] 35049.21
#>
#> $fit.h1$AICc
#> [1] 35050.22
#>
#> $fit.h1$BIC
#> [1] 35222.84
#>
#> $fit.h1$aBIC
#> [1] 35124.35
#>
#> $fit.h1$RMSEA
#> NULL
#>
#> $fit.h1$RMSEA.lower
#> NULL
#>
#> $fit.h1$RMSEA.upper
#> NULL
#>
#> $fit.h1$RMSEA.ci.level
#> NULL
#>
#> $fit.h1$RMSEA.pvalue
#> NULL
#>
#> $fit.h1$RMSEA.close.h0
#> NULL
#>
#>
#> $comparative.fit
#> $comparative.fit$D
#> [1] 676.5418
#>
#> $comparative.fit$df
#> [1] 1
#>
#> $comparative.fit$p
#> [1] 3.775457e-149
#>
#> $comparative.fit$diff.loglik
#> [1] 338.2709
References
Klein, A., & Moosbrugger, H. (2000).
<doi:10.1007/BF02296338>.
"Maximum likelihood estimation of latent interaction effects with the LMS method."
Klein, A. G., & Muthén, B. O. (2007).
<doi:10.1080/00273170701710205>.
"Quasi-maximum likelihood estimation of structural equation models with multiple interaction and quadratic effects."