Skip to contents

fit measures for LMS and QML

fit_measures_da.Rmd

Introduction

This vignette demonstrates how to evaluate and compare model fit for latent interaction models estimated via

  • LMS (Latent Moderated Structural Equations)
  • QML (Quasi-Maximum Likelihood)

using the modsem package (v≥1.0.8). Because standard Chi-square statistics are not available under LMS/QML, we assess fit by:

  1. Examining fit indices for the baseline (no-interaction) model.
  2. Conducting a likelihood-ratio difference test to compare the baseline and interaction models (Klein & Moosbrugger, 2000; Klein & Múthen, 2007).

If the baseline model shows acceptable fit and adding the latent interaction significantly improves fit, the interaction model can also be deemed well-fitting.

Example

We define a model with three latent variables (X, Y, Z) and their interaction (X:Z):

m1 <- "
# Outer (measurement) model
X =~ x1 + x2 + x3
Y =~ y1 + y2 + y3
Z =~ z1 + z2 + z3

# Inner (structural) model
Y ~ X + Z + X:Z
"

# Estimate the full (H1) model via LMS
est_h1 <- modsem(m1, oneInt, method = "lms")

# Estimate the baseline (H0) model without interaction
est_h0 <- estimate_h0(est_h1, calc.se = FALSE) # std.errors are not needed

Fit measures baseline model

To get fit measures for the baseline model you can use the fit_modsem_da() function.

fit_modsem_da(est_h0)
#> $sigma.observed
#>           x1        x2        x3        z1        z2        z3        y1
#> x1 1.1414140 0.7883687 0.8995307 0.2101059 0.1740044 0.1816761 0.7884435
#> x2 0.7883687 0.7969205 0.7227575 0.1611022 0.1303270 0.1323003 0.6405208
#> x3 0.8995307 0.7227575 0.9858399 0.1813015 0.1483665 0.1513961 0.7143555
#> z1 0.2101059 0.1611022 0.1813015 1.1848928 0.8251779 0.8974163 0.7336206
#> z2 0.1740044 0.1303270 0.1483665 0.8251779 0.8282354 0.7268682 0.6114440
#> z3 0.1816761 0.1323003 0.1513961 0.8974163 0.7268682 0.9487013 0.6578085
#> y1 0.7884435 0.6405208 0.7143555 0.7336206 0.6114440 0.6578085 2.6478088
#> y2 0.6253530 0.5123430 0.5569065 0.6010575 0.4998194 0.5413313 1.9878980
#> y3 0.7105927 0.5762476 0.6417552 0.6575885 0.5520473 0.5975011 2.2378289
#>           y2        y3
#> x1 0.6253530 0.7105927
#> x2 0.5123430 0.5762476
#> x3 0.5569065 0.6417552
#> z1 0.6010575 0.6575885
#> z2 0.4998194 0.5520473
#> z3 0.5413313 0.5975011
#> y1 1.9878980 2.2378289
#> y2 1.7403428 1.7841731
#> y3 1.7841731 2.1756928
#> 
#> $sigma.expected
#>           x1        x2        x3        z1        z2        z3        y1
#> x1 1.1408432 0.7887122 0.8984977 0.2011156 0.1631816 0.1774521 0.7860620
#> x2 0.7887122 0.7965219 0.7223142 0.1616795 0.1311839 0.1426561 0.6319257
#> x3 0.8984977 0.7223142 0.9853469 0.1841846 0.1494441 0.1625132 0.7198871
#> z1 0.2011156 0.1616795 0.1841846 1.1843003 0.8244128 0.8965091 0.7493786
#> z2 0.1631816 0.1311839 0.1494441 0.8244128 0.8278212 0.7274115 0.6080325
#> z3 0.1774521 0.1426561 0.1625132 0.8965091 0.7274115 0.9482269 0.6612059
#> y1 0.7860620 0.6319257 0.7198871 0.7493786 0.6080325 0.6612059 2.6464848
#> y2 0.6269615 0.5040227 0.5741806 0.5977030 0.4849655 0.5273766 1.9868791
#> y3 0.7056902 0.5673138 0.6462815 0.6727576 0.5458636 0.5936002 2.2363752
#>           y2        y3
#> x1 0.6269615 0.7056902
#> x2 0.5040227 0.5673138
#> x3 0.5741806 0.6462815
#> z1 0.5977030 0.6727576
#> z2 0.4849655 0.5458636
#> z3 0.5273766 0.5936002
#> y1 1.9868791 2.2363752
#> y2 1.7394726 1.7837286
#> y3 1.7837286 2.1746048
#> 
#> $mu.observed
#>        x1        x2        x3        z1        z2        z3        y1        y2 
#> 1.0228543 1.2154912 0.9193956 1.0113688 1.2055455 0.9155865 1.1802667 1.3346201 
#>        y3 
#> 1.0826391 
#> 
#> $mu.expected
#>            1
#> x1 1.0228543
#> x2 1.2154912
#> x3 0.9193956
#> z1 1.0113688
#> z2 1.2055455
#> z3 0.9155865
#> y1 1.1802667
#> y2 1.3346201
#> y3 1.0826391
#> 
#> $chisq.value
#> [1] 17.5224
#> 
#> $chisq.pvalue
#> [1] 0.8255439
#> 
#> $chisq.df
#> [1] 24
#> 
#> $AIC
#> [1] 35723.75
#> 
#> $AICc
#> [1] 35724.69
#> 
#> $BIC
#> [1] 35891.78
#> 
#> $aBIC
#> [1] 35796.47
#> 
#> $RMSEA
#> [1] 0
#> 
#> $RMSEA.lower
#> [1] 0
#> 
#> $RMSEA.upper
#> [1] 0.01122746
#> 
#> $RMSEA.ci.level
#> [1] 0.9
#> 
#> $RMSEA.pvalue
#> [1] 1
#> 
#> $RMSEA.close.h0
#> [1] 0.05

It can also be used to get fit measures for the full model, but should be pared with chisq = FALSE to avoid the Chi-square test. If it is set to TRUE it will calculate the Chi-square test while ignoring the interaction terms in the model.

fit_modsem_da(est_h1, chisq = FALSE)
#> $sigma.observed
#>           x1        x2        x3        z1        z2        z3        y1
#> x1 1.1414140 0.7883687 0.8995307 0.2101059 0.1740044 0.1816761 0.7884435
#> x2 0.7883687 0.7969205 0.7227575 0.1611022 0.1303270 0.1323003 0.6405208
#> x3 0.8995307 0.7227575 0.9858399 0.1813015 0.1483665 0.1513961 0.7143555
#> z1 0.2101059 0.1611022 0.1813015 1.1848928 0.8251779 0.8974163 0.7336206
#> z2 0.1740044 0.1303270 0.1483665 0.8251779 0.8282354 0.7268682 0.6114440
#> z3 0.1816761 0.1323003 0.1513961 0.8974163 0.7268682 0.9487013 0.6578085
#> y1 0.7884435 0.6405208 0.7143555 0.7336206 0.6114440 0.6578085 2.6478088
#> y2 0.6253530 0.5123430 0.5569065 0.6010575 0.4998194 0.5413313 1.9878980
#> y3 0.7105927 0.5762476 0.6417552 0.6575885 0.5520473 0.5975011 2.2378289
#>           y2        y3
#> x1 0.6253530 0.7105927
#> x2 0.5123430 0.5762476
#> x3 0.5569065 0.6417552
#> z1 0.6010575 0.6575885
#> z2 0.4998194 0.5520473
#> z3 0.5413313 0.5975011
#> y1 1.9878980 2.2378289
#> y2 1.7403428 1.7841731
#> y3 1.7841731 2.1756928
#> 
#> $sigma.expected
#> NULL
#> 
#> $mu.observed
#>        x1        x2        x3        z1        z2        z3        y1        y2 
#> 1.0228543 1.2154912 0.9193956 1.0113688 1.2055455 0.9155865 1.1802667 1.3346201 
#>        y3 
#> 1.0826391 
#> 
#> $mu.expected
#> NULL
#> 
#> $chisq.value
#> NULL
#> 
#> $chisq.pvalue
#> NULL
#> 
#> $chisq.df
#> NULL
#> 
#> $AIC
#> [1] 29437.73
#> 
#> $AICc
#> [1] 29438.73
#> 
#> $BIC
#> [1] 29611.35
#> 
#> $aBIC
#> [1] 29512.86
#> 
#> $RMSEA
#> NULL
#> 
#> $RMSEA.lower
#> NULL
#> 
#> $RMSEA.upper
#> NULL
#> 
#> $RMSEA.ci.level
#> NULL
#> 
#> $RMSEA.pvalue
#> NULL
#> 
#> $RMSEA.close.h0
#> NULL

Difference Test of Fit

Compare H0 vs. H1 using a log-likelihood ratio test:

compare_fit(est_h0, est_h1)
#> $D
#> [1] 6288.025
#> 
#> $df
#> [1] 1
#> 
#> $p
#> [1] 0
#> 
#> $llChange
#> [1] 3144.012

A significant p-value indicates the latent interaction term significantly improves model fit.

Inspecting Fit Indices

For convenience, you can also use the modsem_inspect() function with what = "fit" to get fit indices for both models, and comparative fit in one go.

modsem_inspect(est_h1, what = "fit")
#> $fit.h0
#> $fit.h0$sigma.observed
#>           x1        x2        x3        z1        z2        z3        y1
#> x1 1.1414140 0.7883687 0.8995307 0.2101059 0.1740044 0.1816761 0.7884435
#> x2 0.7883687 0.7969205 0.7227575 0.1611022 0.1303270 0.1323003 0.6405208
#> x3 0.8995307 0.7227575 0.9858399 0.1813015 0.1483665 0.1513961 0.7143555
#> z1 0.2101059 0.1611022 0.1813015 1.1848928 0.8251779 0.8974163 0.7336206
#> z2 0.1740044 0.1303270 0.1483665 0.8251779 0.8282354 0.7268682 0.6114440
#> z3 0.1816761 0.1323003 0.1513961 0.8974163 0.7268682 0.9487013 0.6578085
#> y1 0.7884435 0.6405208 0.7143555 0.7336206 0.6114440 0.6578085 2.6478088
#> y2 0.6253530 0.5123430 0.5569065 0.6010575 0.4998194 0.5413313 1.9878980
#> y3 0.7105927 0.5762476 0.6417552 0.6575885 0.5520473 0.5975011 2.2378289
#>           y2        y3
#> x1 0.6253530 0.7105927
#> x2 0.5123430 0.5762476
#> x3 0.5569065 0.6417552
#> z1 0.6010575 0.6575885
#> z2 0.4998194 0.5520473
#> z3 0.5413313 0.5975011
#> y1 1.9878980 2.2378289
#> y2 1.7403428 1.7841731
#> y3 1.7841731 2.1756928
#> 
#> $fit.h0$sigma.expected
#>           x1        x2        x3        z1        z2        z3        y1
#> x1 1.1408432 0.7887122 0.8984977 0.2011156 0.1631816 0.1774521 0.7860620
#> x2 0.7887122 0.7965219 0.7223142 0.1616795 0.1311839 0.1426561 0.6319257
#> x3 0.8984977 0.7223142 0.9853469 0.1841846 0.1494441 0.1625132 0.7198871
#> z1 0.2011156 0.1616795 0.1841846 1.1843003 0.8244128 0.8965091 0.7493786
#> z2 0.1631816 0.1311839 0.1494441 0.8244128 0.8278212 0.7274115 0.6080325
#> z3 0.1774521 0.1426561 0.1625132 0.8965091 0.7274115 0.9482269 0.6612059
#> y1 0.7860620 0.6319257 0.7198871 0.7493786 0.6080325 0.6612059 2.6464848
#> y2 0.6269615 0.5040227 0.5741806 0.5977030 0.4849655 0.5273766 1.9868791
#> y3 0.7056902 0.5673138 0.6462815 0.6727576 0.5458636 0.5936002 2.2363752
#>           y2        y3
#> x1 0.6269615 0.7056902
#> x2 0.5040227 0.5673138
#> x3 0.5741806 0.6462815
#> z1 0.5977030 0.6727576
#> z2 0.4849655 0.5458636
#> z3 0.5273766 0.5936002
#> y1 1.9868791 2.2363752
#> y2 1.7394726 1.7837286
#> y3 1.7837286 2.1746048
#> 
#> $fit.h0$mu.observed
#>        x1        x2        x3        z1        z2        z3        y1        y2 
#> 1.0228543 1.2154912 0.9193956 1.0113688 1.2055455 0.9155865 1.1802667 1.3346201 
#>        y3 
#> 1.0826391 
#> 
#> $fit.h0$mu.expected
#>            1
#> x1 1.0228543
#> x2 1.2154912
#> x3 0.9193956
#> z1 1.0113688
#> z2 1.2055455
#> z3 0.9155865
#> y1 1.1802667
#> y2 1.3346201
#> y3 1.0826391
#> 
#> $fit.h0$chisq.value
#> [1] 17.5224
#> 
#> $fit.h0$chisq.pvalue
#> [1] 0.8255439
#> 
#> $fit.h0$chisq.df
#> [1] 24
#> 
#> $fit.h0$AIC
#> [1] 35723.75
#> 
#> $fit.h0$AICc
#> [1] 35724.69
#> 
#> $fit.h0$BIC
#> [1] 35891.78
#> 
#> $fit.h0$aBIC
#> [1] 35796.47
#> 
#> $fit.h0$RMSEA
#> [1] 0
#> 
#> $fit.h0$RMSEA.lower
#> [1] 0
#> 
#> $fit.h0$RMSEA.upper
#> [1] 0.01122746
#> 
#> $fit.h0$RMSEA.ci.level
#> [1] 0.9
#> 
#> $fit.h0$RMSEA.pvalue
#> [1] 1
#> 
#> $fit.h0$RMSEA.close.h0
#> [1] 0.05
#> 
#> 
#> $fit.h1
#> $fit.h1$sigma.observed
#>           x1        x2        x3        z1        z2        z3        y1
#> x1 1.1414140 0.7883687 0.8995307 0.2101059 0.1740044 0.1816761 0.7884435
#> x2 0.7883687 0.7969205 0.7227575 0.1611022 0.1303270 0.1323003 0.6405208
#> x3 0.8995307 0.7227575 0.9858399 0.1813015 0.1483665 0.1513961 0.7143555
#> z1 0.2101059 0.1611022 0.1813015 1.1848928 0.8251779 0.8974163 0.7336206
#> z2 0.1740044 0.1303270 0.1483665 0.8251779 0.8282354 0.7268682 0.6114440
#> z3 0.1816761 0.1323003 0.1513961 0.8974163 0.7268682 0.9487013 0.6578085
#> y1 0.7884435 0.6405208 0.7143555 0.7336206 0.6114440 0.6578085 2.6478088
#> y2 0.6253530 0.5123430 0.5569065 0.6010575 0.4998194 0.5413313 1.9878980
#> y3 0.7105927 0.5762476 0.6417552 0.6575885 0.5520473 0.5975011 2.2378289
#>           y2        y3
#> x1 0.6253530 0.7105927
#> x2 0.5123430 0.5762476
#> x3 0.5569065 0.6417552
#> z1 0.6010575 0.6575885
#> z2 0.4998194 0.5520473
#> z3 0.5413313 0.5975011
#> y1 1.9878980 2.2378289
#> y2 1.7403428 1.7841731
#> y3 1.7841731 2.1756928
#> 
#> $fit.h1$sigma.expected
#> NULL
#> 
#> $fit.h1$mu.observed
#>        x1        x2        x3        z1        z2        z3        y1        y2 
#> 1.0228543 1.2154912 0.9193956 1.0113688 1.2055455 0.9155865 1.1802667 1.3346201 
#>        y3 
#> 1.0826391 
#> 
#> $fit.h1$mu.expected
#> NULL
#> 
#> $fit.h1$chisq.value
#> NULL
#> 
#> $fit.h1$chisq.pvalue
#> NULL
#> 
#> $fit.h1$chisq.df
#> NULL
#> 
#> $fit.h1$AIC
#> [1] 29437.73
#> 
#> $fit.h1$AICc
#> [1] 29438.73
#> 
#> $fit.h1$BIC
#> [1] 29611.35
#> 
#> $fit.h1$aBIC
#> [1] 29512.86
#> 
#> $fit.h1$RMSEA
#> NULL
#> 
#> $fit.h1$RMSEA.lower
#> NULL
#> 
#> $fit.h1$RMSEA.upper
#> NULL
#> 
#> $fit.h1$RMSEA.ci.level
#> NULL
#> 
#> $fit.h1$RMSEA.pvalue
#> NULL
#> 
#> $fit.h1$RMSEA.close.h0
#> NULL
#> 
#> 
#> $comparative.fit
#> $comparative.fit$D
#> [1] 6288.025
#> 
#> $comparative.fit$df
#> [1] 1
#> 
#> $comparative.fit$p
#> [1] 0
#> 
#> $comparative.fit$llChange
#> [1] 3144.012

References 

Klein, A., & Moosbrugger, H. (2000). 
  <doi:10.1007/BF02296338>.
  "Maximum likelihood estimation of latent interaction effects with the LMS method."
Klein, A. G., & Muthén, B. O. (2007). 
  <doi:10.1080/00273170701710205>.
  "Quasi-maximum likelihood estimation of structural equation models with multiple interaction and quadratic effects."