Bifactor ESEM Tutorial

Background

Within relational turbulence theory (RTT; Solomon et al., 2016), relational uncertainty is conceptualized as three facets of doubts and insecurities one experiences within a romantic association related to the self, the partner, and the relationship. Conventionally, RTT recognizes self and partner uncertainty as antecedents of relationship uncertainty, which in turn leads to a global characterization of the relationship as turbulent, and this causal model was supported by meta-analytic evidence (Goodboy et al., 2020). However, the correlations among these variables are consistently strong and positive, indicating that they may be influenced by a common factor. In Goodboy et al. (2021), they constrasted multiple measurement models for relational uncertainty with two samples from prior studies, and found the bifactor exploratory structural equation model to be the superior measurement model for the construct (data and Mplus code are available as supplemental materials of this article). I replicate these models in R using the lavaan package with their college sample data.

Goodboy, A. K., Bolkan, S., Brisini, K., & Solomon, D. H. (2021). Relational uncertainty within relational turbulence theory: The bifactor exploratory structural equation model. Journal of Communication, 71(3), 403-430. https://doi.org/10.1093/joc/jqab009

Goodboy, A. K., Bolkan, S., Sharabi, L. L., Myers, S. A., & Baker, J. P. (2020). The relational turbulence model: A meta-analytic review. Human Communication Research, 46(2-3), 222-249.

Solomon, D. H., Knobloch, L. K., Theiss, J. A., & McLaren, R. M. (2016). Relational turbulence theory: Explaining variation in subjective experiences and communication within romantic relationships. Human Communication Research, 42(4), 507-532.

Prep work

Before you start, make sure you placed this .rmd file and “College Sample JOC.dat” in the same folder, so R can find and save any output files to the same place.

Reading the data into the environment.

# Reading the college sample data
college_sample<-read.csv("College-Sample-JOC.csv", header = FALSE)

# Because the data set does not contain variable names, 
# we will label variables here
colnames(college_sample)<-c(paste0("Self",1:6), # Self uncertainty
                            paste0("Part",1:6), # Partner uncertainty
                            paste0("Rel",1:6),  # Relationship uncertainty
                            paste0("Disc",1:4), # Disclosure to network
                            paste0("Lone",1:4), # Loneliness
                            paste0("Turb",1:4)) # Relational turbulence

# Coding missing data
college_sample[college_sample == -99]<-NA

# Subset the data (we only need the 18 relational uncertainty items)
college_ru<-college_sample[1:18]

# View the first 10 rows of college_ru
head(college_ru,10)

##    Self1 Self2 Self3 Self4 Self5 Self6 Part1 Part2 Part3 Part4 Part5 Part6 Rel1
## 1      3     4     5     4     4     5     5     5     4     4     5     4    4
## 2      1     1     1     1     2     1     2     2     2     2     1     2    2
## 3      1     1     1     1     1     1     4     4     4     4     4     3    3
## 4      1     1     1     1     1     1     1     1     1     1     1     1    1
## 5      1     1     1     1     1     1     1     1     1     1     1     1    1
## 6      1     1     1     1     1     1     1     1     1     1     1     1    1
## 7      1     2     1     1     2     1     2     1     1     2     1     2    1
## 8      1     1     1     1     1     1     2     2     1     2     1     1    1
## 9      1     1     1     1     1     1     1     1     1     1     1     1    1
## 10     1     1     2     1     1     1     6     6     6     6     1     6    6
##    Rel2 Rel3 Rel4 Rel5 Rel6
## 1     4    4    3    4    4
## 2     1    1    2    2    1
## 3     1    1    1    4    1
## 4     1    1    6    1    1
## 5     1    1    1    1    1
## 6     1    1    1    1    1
## 7     3    1    1    1    1
## 8     1    1    1    1    1
## 9     1    1    1    1    1
## 10    6    5    1    6    4

Loading R packages

RStudio may prompt you at the top of this panel to install packages that you have not used before. You can also install packages manually by typing in the console: for example, install.packages(“dplyr”).

library(devtools)    # For version control
library(dplyr)       # For data management 
library(GPArotation) # For rotation used in factor analysis
library(lavaan)      # For SEM
library(lavaanPlot)  # For plotting SEM
library(psych)       # For data descriptives and reliability statistics
library(reshape2)    # For data manipulation
library(rlang)       # For string manipulation
library(semTools)    # For factor analysis

Data descriptives and reliabilities

We can first take a look at the item-level descriptives and inter-item correlations.

# Running descriptives
describe(college_ru)

##       vars   n mean   sd median trimmed  mad min max range skew kurtosis   se
## Self1    1 512 3.04 1.62      3    2.97 1.48   1   6     5 0.15    -1.25 0.07
## Self2    2 512 2.75 1.54      2    2.65 1.48   1   6     5 0.31    -1.22 0.07
## Self3    3 512 2.53 1.48      2    2.38 1.48   1   6     5 0.67    -0.77 0.07
## Self4    4 512 2.40 1.43      2    2.22 1.48   1   6     5 0.77    -0.51 0.06
## Self5    5 512 2.28 1.43      2    2.06 1.48   1   6     5 0.90    -0.31 0.06
## Self6    6 512 2.38 1.45      2    2.19 1.48   1   6     5 0.75    -0.60 0.06
## Part1    7 512 2.96 1.59      3    2.86 1.48   1   6     5 0.29    -1.09 0.07
## Part2    8 512 2.80 1.54      2    2.69 1.48   1   6     5 0.35    -1.09 0.07
## Part3    9 511 2.74 1.53      2    2.61 1.48   1   6     5 0.44    -0.98 0.07
## Part4   10 510 2.71 1.48      2    2.59 1.48   1   6     5 0.46    -0.87 0.07
## Part5   11 511 2.50 1.51      2    2.33 1.48   1   6     5 0.71    -0.69 0.07
## Part6   12 512 2.61 1.54      2    2.45 1.48   1   6     5 0.59    -0.81 0.07
## Rel1    13 512 2.78 1.55      3    2.66 1.48   1   6     5 0.41    -0.98 0.07
## Rel2    14 512 3.20 1.62      3    3.16 1.48   1   6     5 0.01    -1.24 0.07
## Rel3    15 512 2.38 1.40      2    2.19 1.48   1   6     5 0.81    -0.33 0.06
## Rel4    16 512 2.31 1.38      2    2.12 1.48   1   6     5 0.85    -0.24 0.06
## Rel5    17 512 2.77 1.57      3    2.62 1.48   1   6     5 0.51    -0.84 0.07
## Rel6    18 512 2.41 1.43      2    2.22 1.48   1   6     5 0.74    -0.51 0.06

# Bivariate correlations among items
round(cor(college_ru, use = "pairwise.complete.obs"),2)

##       Self1 Self2 Self3 Self4 Self5 Self6 Part1 Part2 Part3 Part4 Part5 Part6
## Self1  1.00  0.77  0.61  0.61  0.50  0.58  0.52  0.47  0.41  0.43  0.35  0.44
## Self2  0.77  1.00  0.65  0.71  0.60  0.66  0.50  0.51  0.39  0.45  0.38  0.44
## Self3  0.61  0.65  1.00  0.69  0.62  0.61  0.50  0.50  0.53  0.51  0.46  0.49
## Self4  0.61  0.71  0.69  1.00  0.67  0.66  0.47  0.48  0.39  0.44  0.41  0.43
## Self5  0.50  0.60  0.62  0.67  1.00  0.69  0.41  0.43  0.41  0.38  0.48  0.41
## Self6  0.58  0.66  0.61  0.66  0.69  1.00  0.46  0.47  0.37  0.40  0.40  0.44
## Part1  0.52  0.50  0.50  0.47  0.41  0.46  1.00  0.83  0.67  0.73  0.57  0.68
## Part2  0.47  0.51  0.50  0.48  0.43  0.47  0.83  1.00  0.73  0.81  0.62  0.75
## Part3  0.41  0.39  0.53  0.39  0.41  0.37  0.67  0.73  1.00  0.73  0.65  0.68
## Part4  0.43  0.45  0.51  0.44  0.38  0.40  0.73  0.81  0.73  1.00  0.59  0.80
## Part5  0.35  0.38  0.46  0.41  0.48  0.40  0.57  0.62  0.65  0.59  1.00  0.61
## Part6  0.44  0.44  0.49  0.43  0.41  0.44  0.68  0.75  0.68  0.80  0.61  1.00
## Rel1   0.50  0.54  0.57  0.55  0.52  0.53  0.62  0.71  0.67  0.73  0.60  0.77
## Rel2   0.52  0.55  0.49  0.49  0.46  0.49  0.61  0.62  0.58  0.64  0.49  0.64
## Rel3   0.46  0.50  0.55  0.55  0.55  0.52  0.54  0.57  0.47  0.53  0.46  0.54
## Rel4   0.37  0.38  0.44  0.47  0.46  0.47  0.44  0.49  0.48  0.48  0.46  0.49
## Rel5   0.38  0.35  0.42  0.36  0.33  0.33  0.60  0.67  0.63  0.70  0.52  0.72
## Rel6   0.39  0.42  0.49  0.50  0.46  0.47  0.47  0.50  0.57  0.53  0.48  0.49
##       Rel1 Rel2 Rel3 Rel4 Rel5 Rel6
## Self1 0.50 0.52 0.46 0.37 0.38 0.39
## Self2 0.54 0.55 0.50 0.38 0.35 0.42
## Self3 0.57 0.49 0.55 0.44 0.42 0.49
## Self4 0.55 0.49 0.55 0.47 0.36 0.50
## Self5 0.52 0.46 0.55 0.46 0.33 0.46
## Self6 0.53 0.49 0.52 0.47 0.33 0.47
## Part1 0.62 0.61 0.54 0.44 0.60 0.47
## Part2 0.71 0.62 0.57 0.49 0.67 0.50
## Part3 0.67 0.58 0.47 0.48 0.63 0.57
## Part4 0.73 0.64 0.53 0.48 0.70 0.53
## Part5 0.60 0.49 0.46 0.46 0.52 0.48
## Part6 0.77 0.64 0.54 0.49 0.72 0.49
## Rel1  1.00 0.75 0.59 0.51 0.71 0.53
## Rel2  0.75 1.00 0.52 0.44 0.59 0.47
## Rel3  0.59 0.52 1.00 0.56 0.53 0.48
## Rel4  0.51 0.44 0.56 1.00 0.53 0.62
## Rel5  0.71 0.59 0.53 0.53 1.00 0.53
## Rel6  0.53 0.47 0.48 0.62 0.53 1.00

We can examine the coefficient omega for each subscale. All three scales were reliable.

# Self uncertainty
omega(college_ru[1:6], nfactors = 1)$omega.tot

## [1] 0.9154015

# Partner uncertainty
omega(college_ru[7:12], nfactors = 1)$omega.tot

## [1] 0.9334561

# Relationship uncertainty
omega(college_ru[13:18], nfactors = 1)$omega.tot

## [1] 0.8834218

Fitting measurement models

Goodboy et al. (2021) fitted five such models, and we will replicate them here:

Unidimensional CFA
Independent Clusters Model-CFA (3-Factor CFA)
Exploratory Structural Equation Model
Bifactor CFA
Bifactor ESEM

Unidimensional CFA

This model considers relational uncertainty to be one unidimensional construct, and all 18 items serve as equivalent indicators for this construct.

First, we write out the model in lavaan’s language.

=~ represents “measured by”
~ represents “regressed on”
~~ represents “correlated with”

Here, we specify a latent variable relunc, and that it is measured by all 18 variables in the scale.

# Building the model
UNI.CFA <- 'relunc =~ Self1 + Self2 + Self3 + Self4 + Self5 + Self6
                    + Part1 + Part2 + Part3 + Part4 + Part5 + Part6 
                    + Rel1 + Rel2 + Rel3 + Rel4 + Rel5 + Rel6'

# We run the model and save the results into fit.UNI.CFA
fit.UNI.CFA <- cfa(model = UNI.CFA,
                   data = college_ru,
                   # We use the maximum likelihood with robust SE estimator
                   estimator = "MLR",
                   # Full-information maximum likelihood for missingness
                   missing = "FIML") 

# We can also look at the graphic form of the model
lavaanPlot(name = "UnidimensionalCFA",
           model = fit.UNI.CFA)

Next, we extract the fit statistics from the object. Because it gives a lot of different measures, we can prespecify a subset of fit measures and use them to contrast all the models we will run.

We can see the unidimensional CFA does not fit the data well.

# Creating a subset of fit indices we will examine across all models
fit.subset<-c("chisq.scaled","df","pvalue.scaled",
              "rmsea.scaled","rmsea.pvalue.scale",
              "rmsea.ci.lower.scaled","rmsea.ci.upper.scaled",
              "cfi","tli","srmr","aic","bic")

# Extracting fit indices from the first model
fitmeasures(fit.UNI.CFA, fit.subset)

##          chisq.scaled                    df         pvalue.scaled 
##              1266.303               135.000                 0.000 
##          rmsea.scaled rmsea.ci.lower.scaled rmsea.ci.upper.scaled 
##                 0.128                 0.123                 0.133 
##                   cfi                   tli                  srmr 
##                 0.768                 0.737                 0.088 
##                   aic                   bic 
##             27998.329             28227.198

We will use the same workflow to fit the remaining models.

Independent Clusters Model-CFA (3-Factor CFA)

The second model specifies self, partner, and relationship uncertainty as three correlated latent variables, and each is measured by six indicators with no cross-loading. It fits the data a little bit better compared to the unidimensional model.

# Building the model
FA3.CFA <- 'self =~ Self1 + Self2 + Self3 + Self4 + Self5 + Self6
            part =~ Part1 + Part2 + Part3 + Part4 + Part5 + Part6 
            rel  =~ Rel1 + Rel2 + Rel3 + Rel4 + Rel5 + Rel6'

# We run the model and save the results into fit.UNI.CFA
fit.FA3.CFA <- cfa(model = FA3.CFA,
                   data = college_ru,
                   # We use the maximum likelihood with robust SE estimator
                   estimator = "MLR",
                   # Full-information maximum likelihood for missingness
                   missing = "FIML") 

# We can also look at the graphic form of the model
lavaanPlot(name = "IndependentClustersModelCFA",
           model = fit.FA3.CFA)

# Extracting fit indices from the model
fitmeasures(fit.FA3.CFA, fit.subset)

##          chisq.scaled                    df         pvalue.scaled 
##               569.733               132.000                 0.000 
##          rmsea.scaled rmsea.ci.lower.scaled rmsea.ci.upper.scaled 
##                 0.080                 0.075                 0.086 
##                   cfi                   tli                  srmr 
##                 0.909                 0.894                 0.050 
##                   aic                   bic 
##             26949.169             27190.754

Exploratory Structural Equation Model

The third model specifies self, partner, and relationship uncertainty as three correlated latent variables, and all items are allowed to cross-load onto all factors, though the target rotation will keep cross-loadings as small as possible. Fitting an ESEM in R is not as straightforward as in Mplus, and there is not a simple function for it. However, it is doable.

Mateus Silvestrin’s post explains the workflow for running ESEM in R: https://msilvestrin.me/post/esem/. Overall, there are three steps involved:

Running an exploratory factor analysis to obtain the loadings
Specifying the ESEM model using start values
Fitting the ESEM model

Exploratory factor analysis with target rotation (oblique)

# First, we conduct an EFA without any rotation
efa <- efaUnrotate(# Specifying the dataset
                   data = college_ru,
                   # 3 factors
                   nf = 3)

# Next, we construct a target matrix for the rotation
target.mat <- as.matrix(cbind(c(rep(NA,6),rep(0,12)),
                              c(rep(0,6),rep(NA,6),rep(0,6)),
                              c(rep(0,12),rep(NA,6))))

# View the target matrix
target.mat

##       [,1] [,2] [,3]
##  [1,]   NA    0    0
##  [2,]   NA    0    0
##  [3,]   NA    0    0
##  [4,]   NA    0    0
##  [5,]   NA    0    0
##  [6,]   NA    0    0
##  [7,]    0   NA    0
##  [8,]    0   NA    0
##  [9,]    0   NA    0
## [10,]    0   NA    0
## [11,]    0   NA    0
## [12,]    0   NA    0
## [13,]    0    0   NA
## [14,]    0    0   NA
## [15,]    0    0   NA
## [16,]    0    0   NA
## [17,]    0    0   NA
## [18,]    0    0   NA

# Rotate the matrix of EFA loadings 
rotated <- funRotate(efa, 
                     # Oblique target rotation
                     fun = "targetQ", 
                     # Specifying the target matrix
                     Target = target.mat)

# View the standardized loadings
rotated.loadings <- rotated@loading

# To use loadings as starting values for the ESEM, we need to 
# revert them back to the metric of the items. We do so by 
# multiplying the loadings by the items' standard deviations
rotated.loadings.unstd <- rotated.loadings * describe(college_ru)$sd

Writing a function to generate model syntax

Because the model becomes really complex with all the items cross-loading onto each latent variable, we will write a few lines to generate our lavaan model.

# Manipulating the loadings matrix
loadings.esem <- melt(round(rotated.loadings.unstd,3))

# Naming the columns
colnames(loadings.esem) <- c("item", "latent", "value")

# Labeling our latent variables
loadings.esem$latent <- c(rep("self",18),
                          rep("part",18),
                          rep("rel",18))

# Generating the model syntax

# First, we set two anchors for each latent variables: one being the first item 
# in a corresponding subscale, and another not in the same subscale
anchors <- as.data.frame(cbind(latent = c("self","part","rel","self","part","rel"),
                                 item = c("Self1","Part1","Rel1","Part1","Rel1","Self1")))
# View the anchors dataframe
anchors

##   latent  item
## 1   self Self1
## 2   part Part1
## 3    rel  Rel1
## 4   self Part1
## 5   part  Rel1
## 6    rel Self1

# I created a function for generating the model syntax "esem.lavaan.syntax"
esem.lavaan.syntax <- function (loadings.dt, anchors){
  
  # Create the is_anchor variable in the anchors dataframe
  anchors$is.anchor <- 1
  
  # Merge the anchors dataframe with the loadings dataframe, and rearrange it
  loadings.dt <- merge(anchors, loadings.dt, by = c("item","latent"), all.y = TRUE)
  loadings.dt$is.anchor[is.na(loadings.dt$is.anchor)]<-0
  loadings.dt <- arrange(loadings.dt, desc(is.anchor))  
  loadings.dt <- arrange(loadings.dt, latent)

  # Make syntax column per item; syntax is different depending on is.anchor
  loadings.dt <- loadings.dt %>%
    mutate(syntax = ifelse(is.anchor == 1, 
                           paste0(value,"*", item),
                           paste0("start(",value,")*", item)))
  
  # Combine starting/fix values and item names to generate a line
  # for each latent variable
  loadings.combined <- loadings.dt %>%
    group_by(latent) %>%
    summarize(syntaxline = paste(syntax, collapse = "+"))
  syntaxlines <- paste(loadings.combined$latent, "=~", loadings.combined$syntaxline)

  # Add lines to fix latent variable variance to 1
    c(syntaxlines,
    paste(loadings.combined$latent, "~~ 1*", loadings.combined$latent))
  
}

# Let's take a look at the generated syntax
esem.lavaan.syntax(loadings.esem, anchors)

## [1] "part =~ -1.145*Part1+-0.758*Rel1+start(-1.193)*Part2+start(-0.945)*Part3+start(-1.166)*Part4+start(-0.61)*Part5+start(-1.069)*Part6+start(-0.764)*Rel2+start(-0.216)*Rel3+start(-0.098)*Rel4+start(-0.937)*Rel5+start(-0.182)*Rel6+start(-0.28)*Self1+start(-0.133)*Self2+start(-0.059)*Self3+start(0.16)*Self4+start(0.326)*Self5+start(0.185)*Self6"
## [2] "rel =~ 0.482*Rel1+-0.426*Self1+start(0.001)*Part1+start(0.142)*Part2+start(0.46)*Part3+start(0.266)*Part4+start(0.569)*Part5+start(0.395)*Part6+start(0.189)*Rel2+start(0.515)*Rel3+start(0.786)*Rel4+start(0.582)*Rel5+start(0.714)*Rel6+start(-0.396)*Self2+start(0.264)*Self3+start(0.241)*Self4+start(0.555)*Self5+start(0.301)*Self6"            
## [3] "self =~ 0.313*Part1+1.375*Self1+start(0.186)*Part2+start(-0.017)*Part3+start(0.001)*Part4+start(0.082)*Part5+start(0.017)*Part6+start(0.32)*Rel1+start(0.485)*Rel2+start(0.477)*Rel3+start(0.215)*Rel4+start(-0.135)*Rel5+start(0.281)*Rel6+start(1.524)*Self2+start(0.96)*Self3+start(1.138)*Self4+start(0.942)*Self5+start(1.069)*Self6"            
## [4] "part ~~ 1* part"                                                                                                                                                                                                                                                                                                                                      
## [5] "rel ~~ 1* rel"                                                                                                                                                                                                                                                                                                                                        
## [6] "self ~~ 1* self"

The first three lines specify the measurement portion of the model. For example, self =~ 1.375*Self1 fixes Self1’s unstandardized loading on self uncertainty at that value, while self =~ start(0.96)*Self2 gives the loading for Self2 a starting value without fixing it. Lines 4-6 sets the variance of each latent variable at 1.

Fitting the ESEM model

Now that the new function has taken care of the syntax, we can just plug the function along with the input data frames we specified above into the CFA function.

fit.FA3.ESEM <- cfa(model = esem.lavaan.syntax(loadings.esem, anchors),
                    data = college_ru,
                    # We use the maximum likelihood with robust SE estimator
                    estimator = "MLR",
                    # Full-information maximum likelihood for missingness
                    missing = "FIML")

# We can also look at the graphic form of the model
lavaanPlot(name = "ESEM",
           model = fit.FA3.ESEM)

# Let's take a look at the standardized loadings 
select(arrange(arrange(parameterestimates(fit.FA3.ESEM, standardized = T)[1:54,], rhs),lhs),
       # only view a subset of output of interest
       lhs:est,pvalue,std.all)

##     lhs op   rhs    est pvalue std.all
## 1  part =~ Part1 -1.145     NA  -0.720
## 2  part =~ Part2 -1.187  0.000  -0.769
## 3  part =~ Part3 -0.929  0.000  -0.609
## 4  part =~ Part4 -1.165  0.000  -0.790
## 5  part =~ Part5 -0.578  0.003  -0.382
## 6  part =~ Part6 -1.065  0.000  -0.692
## 7  part =~  Rel1 -0.758     NA  -0.490
## 8  part =~  Rel2 -0.774  0.000  -0.479
## 9  part =~  Rel3 -0.193  0.385  -0.137
## 10 part =~  Rel4 -0.059  0.848  -0.043
## 11 part =~  Rel5 -0.927  0.000  -0.591
## 12 part =~  Rel6 -0.155  0.565  -0.108
## 13 part =~ Self1 -0.326  0.191  -0.201
## 14 part =~ Self2 -0.183  0.476  -0.119
## 15 part =~ Self3 -0.055  0.756  -0.037
## 16 part =~ Self4  0.164  0.441   0.115
## 17 part =~ Self5  0.351  0.274   0.245
## 18 part =~ Self6  0.183  0.426   0.126
## 19  rel =~ Part1 -0.001  0.989  -0.001
## 20  rel =~ Part2  0.136  0.219   0.088
## 21  rel =~ Part3  0.461  0.002   0.302
## 22  rel =~ Part4  0.241  0.172   0.164
## 23  rel =~ Part5  0.595  0.003   0.394
## 24  rel =~ Part6  0.375  0.016   0.244
## 25  rel =~  Rel1  0.482     NA   0.312
## 26  rel =~  Rel2  0.172  0.090   0.107
## 27  rel =~  Rel3  0.559  0.067   0.399
## 28  rel =~  Rel4  0.839  0.023   0.609
## 29  rel =~  Rel5  0.559  0.002   0.356
## 30  rel =~  Rel6  0.758  0.020   0.530
## 31  rel =~ Self1 -0.426     NA  -0.263
## 32  rel =~ Self2 -0.387  0.003  -0.252
## 33  rel =~ Self3  0.315  0.319   0.214
## 34  rel =~ Self4  0.306  0.438   0.214
## 35  rel =~ Self5  0.643  0.241   0.449
## 36  rel =~ Self6  0.364  0.395   0.252
## 37 self =~ Part1  0.313     NA   0.197
## 38 self =~ Part2  0.182  0.002   0.118
## 39 self =~ Part3 -0.031  0.820  -0.020
## 40 self =~ Part4  0.005  0.959   0.003
## 41 self =~ Part5  0.057  0.787   0.038
## 42 self =~ Part6  0.015  0.887   0.010
## 43 self =~  Rel1  0.296  0.057   0.192
## 44 self =~  Rel2  0.484  0.000   0.299
## 45 self =~  Rel3  0.437  0.101   0.311
## 46 self =~  Rel4  0.159  0.643   0.115
## 47 self =~  Rel5 -0.148  0.341  -0.094
## 48 self =~  Rel6  0.225  0.471   0.158
## 49 self =~ Self1  1.375     NA   0.849
## 50 self =~ Self2  1.514  0.000   0.983
## 51 self =~ Self3  0.916  0.000   0.620
## 52 self =~ Self4  1.085  0.000   0.758
## 53 self =~ Self5  0.872  0.030   0.609
## 54 self =~ Self6  1.011  0.001   0.699

# Extracting fit indices from the model
fitmeasures(fit.FA3.ESEM, fit.subset)

##          chisq.scaled                    df         pvalue.scaled 
##               362.480               102.000                 0.000 
##          rmsea.scaled rmsea.ci.lower.scaled rmsea.ci.upper.scaled 
##                 0.071                 0.064                 0.077 
##                   cfi                   tli                  srmr 
##                 0.948                 0.922                 0.025 
##                   aic                   bic 
##             26687.249             27055.983

We can see from the standardized loadings that items mainly loaded onto the latent variables they belong to, while cross-loadings are permitted in this model. Note that the loadings for partner uncertainty are negative due to factor indeterminacy. We can reverse the signs on those loadings when we generate the model syntax.

Bifactor CFA

The fourth model specifies a bifactor configuration: a general relational uncertainty factor, and three residualized specific factors corresponding to self, partner, and relationship. This model requires factors be orthogonal to one another.

# Building the model
BIFA.CFA <- 'general =~ NA*Self1 + Self2 + Self3 + Self4 + Self5 + Self6
                      + Part1 + Part2 + Part3 + Part4 + Part5 + Part6 
                      + Rel1 + Rel2 + Rel3 + Rel4 + Rel5 + Rel6
                self =~ NA*Self1 + Self2 + Self3 + Self4 + Self5 + Self6
                part =~ NA*Part1 + Part2 + Part3 + Part4 + Part5 + Part6 
                rel  =~ NA*Rel1 + Rel2 + Rel3 + Rel4 + Rel5 + Rel6
                # Factors are orthogonal
                general ~~ 0*self + 0*part + 0*rel
                self ~~ 0*part + 0*rel
                part ~~ 0*rel
                # Setting latent variable variance at 1
                general ~~ 1*general
                self ~~ 1*self
                part ~~ 1*part
                rel ~~ 1*rel'

# We run the model and save the results into fit.UNI.CFA
fit.BIFA.CFA <- cfa(model = BIFA.CFA,
                   data = college_ru,
                   # We use the maximum likelihood with robust SE estimator
                   estimator = "MLR",
                   # Full-information maximum likelihood for missingness
                   missing = "FIML") 

# We can also look at the graphic form of the model
lavaanPlot(name = "BifactorCFA",
           model = fit.BIFA.CFA)

# Extracting fit indices from the model
fitmeasures(fit.BIFA.CFA, fit.subset)

##          chisq.scaled                    df         pvalue.scaled 
##               443.610               117.000                 0.000 
##          rmsea.scaled rmsea.ci.lower.scaled rmsea.ci.upper.scaled 
##                 0.074                 0.068                 0.080 
##                   cfi                   tli                  srmr 
##                 0.933                 0.912                 0.046 
##                   aic                   bic 
##             26782.997             27088.156

Bifactor ESEM

Here, we will follow the same steps as the previous ESEM. The difference between the two mostly stems from how latent factors are related in these models. In regular ESEM, factors correlate, so we use an oblique rotation for the EFA portion of the process, while bifactor ESEM sets factor correlations at 0, so orthogonal rotation is used in the EFA step.

Exploratory factor analysis with target rotation (orthogonal)

# First, we conduct an EFA without any rotation
efa.bifac <- efaUnrotate(data = college_ru,
                         # 4 factors
                         nf = 4)

# Constructing a target matrix
target.mat.bifac <- as.matrix(cbind(rep(NA,18),
                              c(rep(NA,6),rep(0,12)),
                              c(rep(0,6),rep(NA,6),rep(0,6)),
                              c(rep(0,12),rep(NA,6))))

#View the target matrix
target.mat.bifac

##       [,1] [,2] [,3] [,4]
##  [1,]   NA   NA    0    0
##  [2,]   NA   NA    0    0
##  [3,]   NA   NA    0    0
##  [4,]   NA   NA    0    0
##  [5,]   NA   NA    0    0
##  [6,]   NA   NA    0    0
##  [7,]   NA    0   NA    0
##  [8,]   NA    0   NA    0
##  [9,]   NA    0   NA    0
## [10,]   NA    0   NA    0
## [11,]   NA    0   NA    0
## [12,]   NA    0   NA    0
## [13,]   NA    0    0   NA
## [14,]   NA    0    0   NA
## [15,]   NA    0    0   NA
## [16,]   NA    0    0   NA
## [17,]   NA    0    0   NA
## [18,]   NA    0    0   NA

# Using the target matrix to rotate the loadings
rotated.bifac <- funRotate(# The EFA results to be rotated
                           efa.bifac, 
                           # Target rotation (orthogonal)
                           fun = "targetT", 
                           # Specify the target matrix
                           Target = target.mat.bifac)

# To use loadings as starting values for the ESEM, we need to 
# revert them back to the metric of the items. We do so by 
# multiplying the loadings by the items' standard deviations
rotated.bifac.loadings.unstd <- rotated.bifac@loading*describe(college_ru)$sd


# Manipulate the loading matrix, name columns, label factors
loadings.esem.bifac <- melt(round(rotated.bifac.loadings.unstd,3))
colnames(loadings.esem.bifac) <- c("item","latent","value")
loadings.esem.bifac$latent<-c(rep("general",18),
                                 rep("self",18),
                                 rep("part",18),
                                 rep("rel",18))

# Setting anchors for the bifactor ESEM 
anchors.bifac <- as.data.frame(cbind(latent = c("general","general","general",
                                                "self","part","rel"),
                                     item = c("Self1","Part1","Rel1",
                                              "Self1","Part1","Rel1")))

#Examine the generated syntax
esem.lavaan.syntax(loadings.esem.bifac,anchors.bifac)

## [1] "general =~ 1.135*Part1+1.363*Rel1+0.858*Self1+start(1.224)*Part2+start(1.17)*Part3+start(1.201)*Part4+start(1.062)*Part5+start(1.278)*Part6+start(1.205)*Rel2+start(0.978)*Rel3+start(0.924)*Rel4+start(1.226)*Rel5+start(0.978)*Rel6+start(0.873)*Self2+start(0.978)*Self3+start(0.905)*Self4+start(0.903)*Self5+start(0.887)*Self6"                       
## [2] "part =~ -0.825*Part1+start(-0.752)*Part2+start(-0.401)*Part3+start(-0.505)*Part4+start(-0.228)*Part5+start(-0.304)*Part6+start(0.002)*Rel1+start(-0.122)*Rel2+start(-0.021)*Rel3+start(0.091)*Rel4+start(-0.185)*Rel5+start(0.051)*Rel6+start(-0.209)*Self1+start(-0.137)*Self2+start(0.019)*Self3+start(0.102)*Self4+start(0.242)*Self5+start(0.112)*Self6"
## [3] "rel =~ 0.355*Rel1+start(-0.005)*Part1+start(0.019)*Part2+start(-0.033)*Part3+start(0.162)*Part4+start(-0.169)*Part5+start(0.274)*Part6+start(0.439)*Rel2+start(-0.158)*Rel3+start(-0.316)*Rel4+start(0.195)*Rel5+start(-0.267)*Rel6+start(0.298)*Self1+start(0.242)*Self2+start(-0.106)*Self3+start(-0.145)*Self4+start(-0.335)*Self5+start(-0.184)*Self6"  
## [4] "self =~ 0.958*Self1+start(0.162)*Part1+start(0.032)*Part2+start(-0.139)*Part3+start(-0.135)*Part4+start(-0.052)*Part5+start(-0.166)*Part6+start(0.02)*Rel1+start(0.189)*Rel2+start(0.257)*Rel3+start(0.064)*Rel4+start(-0.281)*Rel5+start(0.109)*Rel6+start(1.069)*Self2+start(0.628)*Self3+start(0.774)*Self4+start(0.625)*Self5+start(0.725)*Self6"       
## [5] "general ~~ 1* general"                                                                                                                                                                                                                                                                                                                                      
## [6] "part ~~ 1* part"                                                                                                                                                                                                                                                                                                                                            
## [7] "rel ~~ 1* rel"                                                                                                                                                                                                                                                                                                                                              
## [8] "self ~~ 1* self"

fit.BIFA.ESEM <- cfa(model = c(esem.lavaan.syntax(loadings.esem.bifac,anchors.bifac),
                               # Adding orthogonal constraints on factors
                               'general ~~ 0*self + 0*part + 0*rel
                                self ~~ 0*part + 0*rel
                                part ~~ 0*rel'),
                    data =  college_ru,
                    # We use the maximum likelihood with robust SE estimator
                    estimator = "MLR",
                    # Full-information meximum likelihood for missingness
                    missing = "FIML")

# We can take a look at the graphic form of the model
lavaanPlot(name = "BifactorESEM",
           model = fit.BIFA.ESEM)

# Examine the loadings 
select(arrange(arrange(parameterestimates(fit.BIFA.ESEM,standardized = T)[1:72,], rhs),lhs),
  # only view a subset of output of interest
  lhs:est,pvalue,std.all)

##        lhs op   rhs    est pvalue std.all
## 1  general =~ Part1  1.135     NA   0.715
## 2  general =~ Part2  1.223  0.000   0.793
## 3  general =~ Part3  1.170  0.000   0.767
## 4  general =~ Part4  1.201  0.000   0.815
## 5  general =~ Part5  1.060  0.000   0.701
## 6  general =~ Part6  1.279  0.000   0.832
## 7  general =~  Rel1  1.363     NA   0.883
## 8  general =~  Rel2  1.201  0.000   0.744
## 9  general =~  Rel3  0.975  0.000   0.695
## 10 general =~  Rel4  0.916  0.000   0.665
## 11 general =~  Rel5  1.219  0.000   0.777
## 12 general =~  Rel6  0.969  0.000   0.678
## 13 general =~ Self1  0.858     NA   0.530
## 14 general =~ Self2  0.873  0.000   0.567
## 15 general =~ Self3  0.973  0.000   0.659
## 16 general =~ Self4  0.900  0.000   0.629
## 17 general =~ Self5  0.899  0.000   0.628
## 18 general =~ Self6  0.885  0.000   0.612
## 19    part =~ Part1 -0.825     NA  -0.519
## 20    part =~ Part2 -0.742  0.000  -0.482
## 21    part =~ Part3 -0.395  0.016  -0.259
## 22    part =~ Part4 -0.493  0.000  -0.335
## 23    part =~ Part5 -0.216  0.283  -0.143
## 24    part =~ Part6 -0.294  0.138  -0.191
## 25    part =~  Rel1  0.014  0.945   0.009
## 26    part =~  Rel2 -0.119  0.729  -0.074
## 27    part =~  Rel3 -0.017  0.900  -0.012
## 28    part =~  Rel4  0.090  0.725   0.065
## 29    part =~  Rel5 -0.178  0.400  -0.114
## 30    part =~  Rel6  0.040  0.852   0.028
## 31    part =~ Self1 -0.212  0.764  -0.131
## 32    part =~ Self2 -0.138  0.850  -0.089
## 33    part =~ Self3  0.015  0.962   0.010
## 34    part =~ Self4  0.099  0.800   0.069
## 35    part =~ Self5  0.240  0.458   0.168
## 36    part =~ Self6  0.107  0.755   0.074
## 37     rel =~ Part1 -0.012  0.986  -0.008
## 38     rel =~ Part2  0.017  0.978   0.011
## 39     rel =~ Part3 -0.035  0.930  -0.023
## 40     rel =~ Part4  0.166  0.715   0.113
## 41     rel =~ Part5 -0.164  0.470  -0.109
## 42     rel =~ Part6  0.272  0.365   0.177
## 43     rel =~  Rel1  0.355     NA   0.230
## 44     rel =~  Rel2  0.427  0.047   0.264
## 45     rel =~  Rel3 -0.163  0.443  -0.116
## 46     rel =~  Rel4 -0.336  0.131  -0.244
## 47     rel =~  Rel5  0.191  0.511   0.122
## 48     rel =~  Rel6 -0.292  0.284  -0.204
## 49     rel =~ Self1  0.285  0.688   0.176
## 50     rel =~ Self2  0.242  0.720   0.157
## 51     rel =~ Self3 -0.118  0.752  -0.080
## 52     rel =~ Self4 -0.157  0.687  -0.110
## 53     rel =~ Self5 -0.330  0.253  -0.230
## 54     rel =~ Self6 -0.179  0.588  -0.124
## 55    self =~ Part1  0.153  0.766   0.096
## 56    self =~ Part2  0.023  0.960   0.015
## 57    self =~ Part3 -0.141  0.603  -0.092
## 58    self =~ Part4 -0.138  0.557  -0.094
## 59    self =~ Part5 -0.048  0.838  -0.031
## 60    self =~ Part6 -0.164  0.128  -0.107
## 61    self =~  Rel1  0.019  0.923   0.013
## 62    self =~  Rel2  0.196  0.251   0.121
## 63    self =~  Rel3  0.261  0.057   0.186
## 64    self =~  Rel4  0.068  0.688   0.049
## 65    self =~  Rel5 -0.280  0.001  -0.179
## 66    self =~  Rel6  0.109  0.514   0.076
## 67    self =~ Self1  0.958     NA   0.592
## 68    self =~ Self2  1.066  0.000   0.693
## 69    self =~ Self3  0.631  0.000   0.427
## 70    self =~ Self4  0.776  0.000   0.542
## 71    self =~ Self5  0.628  0.000   0.439
## 72    self =~ Self6  0.723  0.000   0.500

# Extracting fit indices from the model
fitmeasures(fit.BIFA.ESEM, fit.subset)

##          chisq.scaled                    df         pvalue.scaled 
##               248.553                87.000                 0.000 
##          rmsea.scaled rmsea.ci.lower.scaled rmsea.ci.upper.scaled 
##                 0.060                 0.053                 0.068 
##                   cfi                   tli                  srmr 
##                 0.969                 0.945                 0.021 
##                   aic                   bic 
##             26546.623             26978.932

Notice that these loadings are within rounding errors with the parameters reported in Goodboy et al. (2021). We have successfully replicated the bifactor ESEM model.

Model Evaluation

Next, we can juxtapose the fit statistics of all five models. Again, some of these values slightly differ from Goodboy et al. (2021; Table 1, top panel), but this does not impact the substantive conclusions we can draw from this set of results.

fit.summary <- round(rbind(fitmeasures(fit.UNI.CFA, fit.subset),
                           fitmeasures(fit.FA3.CFA, fit.subset),
                           fitmeasures(fit.FA3.ESEM, fit.subset),
                           fitmeasures(fit.BIFA.CFA, fit.subset),
                           fitmeasures(fit.BIFA.ESEM, fit.subset)),3)

rownames(fit.summary)<-c("Uni CFA",
                         "ICM-CFA",
                         "ESEM",
                         "Bifactor CFA",
                         "Bifactor ESEM")

fit.summary

##               chisq.scaled  df pvalue.scaled rmsea.scaled rmsea.ci.lower.scaled
## Uni CFA           1266.303 135             0        0.128                 0.123
## ICM-CFA            569.733 132             0        0.080                 0.075
## ESEM               362.480 102             0        0.071                 0.064
## Bifactor CFA       443.610 117             0        0.074                 0.068
## Bifactor ESEM      248.553  87             0        0.060                 0.053
##               rmsea.ci.upper.scaled   cfi   tli  srmr      aic      bic
## Uni CFA                       0.133 0.768 0.737 0.088 27998.33 28227.20
## ICM-CFA                       0.086 0.909 0.894 0.050 26949.17 27190.75
## ESEM                          0.077 0.948 0.922 0.025 26687.25 27055.98
## Bifactor CFA                  0.080 0.933 0.912 0.046 26783.00 27088.16
## Bifactor ESEM                 0.068 0.969 0.945 0.021 26546.62 26978.93

Additional Information

I created this tutorial with a system environment and versions of R and packages that might be different from yours. If R reports errors when you attempt to run this tutorial, running the code chunk below and comparing your output and the tutorial posted on the website may be helpful.

session_info(pkgs = c("attached"))

## ─ Session info ───────────────────────────────────────────────────────────────
##  setting  value
##  version  R version 4.2.0 (2022-04-22)
##  os       macOS Big Sur/Monterey 10.16
##  system   x86_64, darwin17.0
##  ui       X11
##  language (EN)
##  collate  en_US.UTF-8
##  ctype    en_US.UTF-8
##  tz       America/New_York
##  date     2022-08-25
##  pandoc   2.18 @ /Applications/RStudio.app/Contents/MacOS/quarto/bin/tools/ (via rmarkdown)
## 
## ─ Packages ───────────────────────────────────────────────────────────────────
##  package     * version  date (UTC) lib source
##  devtools    * 2.4.3    2021-11-30 [1] CRAN (R 4.2.0)
##  dplyr       * 1.0.9    2022-04-28 [1] CRAN (R 4.2.0)
##  GPArotation * 2022.4-1 2022-04-16 [1] CRAN (R 4.2.0)
##  lavaan      * 0.6-12   2022-07-04 [1] CRAN (R 4.2.0)
##  lavaanPlot  * 0.6.2    2021-08-13 [1] CRAN (R 4.2.0)
##  psych       * 2.2.5    2022-05-10 [1] CRAN (R 4.2.0)
##  reshape2    * 1.4.4    2020-04-09 [1] CRAN (R 4.2.0)
##  rlang       * 1.0.2    2022-03-04 [1] CRAN (R 4.2.0)
##  semTools    * 0.5-6    2022-05-10 [1] CRAN (R 4.2.0)
##  usethis     * 2.1.6    2022-05-25 [1] CRAN (R 4.2.0)
## 
##  [1] /Library/Frameworks/R.framework/Versions/4.2/Resources/library
## 
## ──────────────────────────────────────────────────────────────────────────────