Meta-analysis with R package ‘psychmeta’

Author: Yuwei Li
Version date: 8/20/2022

This tutorial was originally written to support Dr. James Dillard’s graduate-level meta-analysis course in Fall 2021. The tutorial uses the Hunter-Schmidt approach to meta-analysis (individual correction of measurement errors), as well as example data from Schmidt & Hunter’s book (2014) to illustrate the procedures of running and plotting meta-analysis on a bivariate correlation, as well as moderation analysis.

Prep work

A handy R shortcut

Keys Function
Ctrl/Cmd + Enter run highlighted/selected code or current line

Setting up packages

  • You only need to install packages once. To run the first line, remove the #, then click the green triangle on top of the code block. The library() function loads packages to the environment for you to use, and needs to be repeated if you reboot your computer/R/RStudio.

  • You can find out information about packages and functions using the help tab in the lower-right panel, or look for their documentations on cran.

#install.packages("metafor","psychmeta","dplyr","readxl")
library(metafor)
library(psychmeta)
library(dplyr)
library(devtools)
library(readxl)

Reading the dataset into the environment

  • If your files are stored in the same directory as your *.rmd file, you can locate them using just quotes ““. Otherwise, you can specify the file path based on your OS.

  • read_xlsx() reads excel spreadsheets. The “1” below identifies the worksheet to be read in an *.xlsx file, and is optional. Here, head() shows the first 6 lines read from the data file.

read_xlsx("ExampleData.xlsx",1)->ES
head(ES)
## # A tibble: 6 × 6
##   `Study No.`  corr    Nr   Rxx   Ryy Moderator
##         <dbl> <dbl> <dbl> <dbl> <dbl>     <dbl>
## 1           1  0.35    68  0.55   0.8         2
## 2           2  0.07   150  0.55   0.6         2
## 3           3  0.11   250  0.55   0.8         2
## 4           4  0.31    68  0.55   0.6         2
## 5           5  0.18   200  0.67   0.8         2
## 6           6  0.36    68  0.67   0.6         2
  • Arrows (-> & <-) are used to define/create objects in R, analogous to “=” in some other programing languages. The following line creates a “dataframe object” named “ES,” and it shows up in the upper-right environment tab.
  • variable.names() gives you a list of column names of a data frame/matrix object. This will come in handy.
variable.names(ES)
## [1] "Study No." "corr"      "Nr"        "Rxx"       "Ryy"       "Moderator"

Running meta-analysis

  • In the psychmeta package, ma_r and ma_d are the generic meta-analysis function for r and d values, respectively. These functions create an object when run, so you should save the object using arrows (see above).
  • Once you run the ma_r() function below, you have your meta-analysis results stored in the object called “meta.object.”
  • If your column/variable names contain illegal characters like space, use backtick ` to wrap around your variable name in the function.
ma_r(rxyi = corr,
     n = Nr,
     sample_id = `Study No.`,
     ma_method = "ic", #individual corrections
     rxx = Rxx, correct_rxx = TRUE, 
     ryy = Ryy, correct_ryy = TRUE,
     data = ES)->meta.object
  • We can view the results of the meta-analysis using individually corrected effect sizes by calling out elements in meta.object. $ extracts parts of an object (analogous to /slashes in file paths). Available elements will pop up as you type $ to the end of an object.
  • Here, we can examine the results table with both measurement errors in x & y corrected.
meta.object$meta_tables$`analysis_id: 1`$individual_correction$true_score
## Meta-analysis table 
##    k    N mean_r  sd_r   se_r sd_res mean_rho sd_r_c se_r_c sd_rho CI_LL_95
## 1 20 3116  0.326 0.207 0.0462  0.194    0.451  0.258 0.0577  0.239     0.33
##   CI_UL_95 CR_LL_80 CR_UL_80
## 1    0.572    0.134    0.768
  • Comparing results between correcting measurement error for zero, one, and two variables:
##                  k    N mean_r var_r var_e var_res  sd_r  se_r  sd_e sd_res
## barebones       20 3116  0.326 0.043 0.005   0.038 0.207 0.046 0.072  0.194
## corrected for x 20 3116  0.326 0.043 0.005   0.038 0.207 0.046 0.072  0.194
## corrected for y 20 3116  0.326 0.043 0.005   0.038 0.207 0.046 0.072  0.194
## corrected both  20 3116  0.326 0.043 0.005   0.038 0.207 0.046 0.072  0.194
##                 CI_LL_95 CI_UL_95 CR_LL_80 CR_UL_80 mean_rho var_r_c var_e_c
## barebones          0.230    0.423    0.069    0.583       NA      NA      NA
## corrected for x    0.286    0.495    0.116    0.665    0.390   0.050   0.007
## corrected for y    0.288    0.499    0.116    0.670    0.393   0.051   0.007
## corrected both     0.330    0.572    0.134    0.768    0.451   0.067   0.009
##                 var_rho sd_r_c se_r_c sd_e_c sd_rho
## barebones            NA     NA     NA     NA     NA
## corrected for x   0.043  0.223  0.050  0.084  0.207
## corrected for y   0.043  0.225  0.050  0.084  0.208
## corrected both    0.057  0.258  0.058  0.097  0.239

Getting heterogeneity information and plots

  • By default, ma_r and ma_d do not contain heterogeneity statistics or plots. You can run the lines below to add those information to meta.object. Now, you can call out desired information the same way you accessed the results table.
heterogeneity(meta.object)->meta.object
plot_forest(meta.object)->meta.object
plot_funnel(meta.object)->meta.object
  • Heterogeneity statistics
meta.object$heterogeneity$`analysis id: 1`$individual_correction$true_score
## 
## Heterogeneity results for r 
## ---------------------------
## 
## Accounted for a total of 14.099% of variance
## 
## Correlation between r values and artifactual perturbations: 0.375
## 
## The reliability of observed effect sizes is: 0.859
## 
## 
## Random effects variance estimates
## ---------------------------------
## Hunter-Schmidt method (with k-correction):
##   sd_rho  (tau):   0.239, SE = 0.049, 95% CI = [0.171, 0.364] 
##   var_rho (tau^2): 0.057, SE = 0.023, 95% CI = [0.029, 0.133] 
## 
##   Q statistic: 134.763 (df = 19, p = 0.000) 
##   H: 2.663   H^2: 7.093   I^2: 85.901
## 
## DerSimonian-Laird method:
##   sd_rho  (tau):   0.243
##   var_rho (tau^2): 0.059
## 
##   Q statistic: 135.327
##   H: 2.669   H^2: 7.122   I^2: 85.960
## 
## Outlier-robust method (absolute deviation from mean):
##   sd_rho  (tau_r):   0.261
##   var_rho (tau_r^2): 0.068
## 
##   Q_r statistic: 42.744
##   H_r: 2.748   H_r^2: 7.552   I_r^2: 0.868
## 
## Outlier-robust method (absolute deviation from median):
##   sd_rho  (tau_m):   0.255
##   var_rho (tau_m^2): 0.065
## 
##   Q_m statistic: 43.052
##   H_m: 2.698   H_m^2: 7.279   I_m^2: 0.863
  • Forest plot
meta.object$forest$`analysis id: 1`$unmoderated$individual_correction$ts

  • Funnel plot
meta.object$funnel$`analysis id: 1`$individual_correction$true_score

Moderation Analysis

  • Moderation analysis requires specification of three more settings – name of the moderator variable, whether the moderator is categorical, and the type of moderation analysis (none, simple, or hierarchical). The object generated will have more sets of results, and they will represent the overall meta-analysis, and then meta-analyses at different levels of the moderator variable.
ma_r(rxyi = corr,
     n = Nr,
     sample_id = `Study No.`,
     ma_method = "ic", #individual corrections
     rxx = Rxx, correct_rxx = TRUE, 
     ryy = Ryy, correct_ryy = TRUE,
     moderators = Moderator,
     cat_moderators = TRUE,
     moderator_type = "simple",
     data = ES)->meta.object2
  • We can add heterogeneity and plots to the object like we did with the unmoderated model.
heterogeneity(meta.object2)->meta.object2
## Heterogeneity analyses have been added to 'ma_obj' - use get_heterogeneity() to retrieve them.
plot_forest(meta.object2)->meta.object2
## Forest plots have been added to 'ma_obj' - use get_plots() to retrieve them.
plot_funnel(meta.object2)->meta.object2
## Funnel plots have been added to 'ma_obj' - use get_plots() to retrieve them.
  • Calling out “Moderator” within the object give you the identifiers of multiple analyses in this object. Here, we take a look at the unmoderated and moderated effects.
meta.object2$Moderator
## analysis_id: 1 analysis_id: 2 analysis_id: 3 
##     All Levels              1              2 
## Levels: All Levels 1 2
rbind(meta.object2$meta_tables$`analysis_id: 1`$individual_correction$true_score,
      meta.object2$meta_tables$`analysis_id: 2`$individual_correction$true_score,
      meta.object2$meta_tables$`analysis_id: 3`$individual_correction$true_score)
## Meta-analysis table 
##    k    N mean_r   sd_r   se_r sd_res mean_rho sd_r_c se_r_c sd_rho CI_LL_95
## 1 20 3116  0.326 0.2066 0.0462 0.1937    0.451  0.258 0.0577 0.2391    0.330
## 2  7 1100  0.575 0.0830 0.0314 0.0633    0.730  0.101 0.0382 0.0747    0.637
## 3 13 2016  0.191 0.0889 0.0247 0.0432    0.267  0.122 0.0338 0.0559    0.193
##   CI_UL_95 CR_LL_80 CR_UL_80
## 1    0.572    0.134    0.768
## 2    0.824    0.623    0.838
## 3    0.341    0.191    0.343
  • Heterogeneity (not shown in the knitted file, runnable in rmd)
meta.object2$heterogeneity
  • Forest plot with moderation
meta.object2$forest$`analysis id: 1`$moderated$individual_correction$ts

  • Note: Funnel plots are produced for individual analyses in psychmeta and do not show moderation.

Additional Information

I created this tutorial with a system environment and versions of R and packages that might be different from yours. If R reports errors when you attempt to run this tutorial, running the code chunk below and comparing your output and the tutorial posted on the website may be helpful.

session_info(pkgs = c("attached"))
## ─ Session info ───────────────────────────────────────────────────────────────
##  setting  value
##  version  R version 4.2.0 (2022-04-22)
##  os       macOS Big Sur/Monterey 10.16
##  system   x86_64, darwin17.0
##  ui       X11
##  language (EN)
##  collate  en_US.UTF-8
##  ctype    en_US.UTF-8
##  tz       America/New_York
##  date     2022-08-20
##  pandoc   2.18 @ /Applications/RStudio.app/Contents/MacOS/quarto/bin/tools/ (via rmarkdown)
## 
## ─ Packages ───────────────────────────────────────────────────────────────────
##  package   * version date (UTC) lib source
##  devtools  * 2.4.3   2021-11-30 [1] CRAN (R 4.2.0)
##  dplyr     * 1.0.9   2022-04-28 [1] CRAN (R 4.2.0)
##  Matrix    * 1.4-1   2022-03-23 [1] CRAN (R 4.2.0)
##  metadat   * 1.2-0   2022-04-06 [1] CRAN (R 4.2.0)
##  metafor   * 3.4-0   2022-04-21 [1] CRAN (R 4.2.0)
##  psychmeta * 2.6.4   2022-07-11 [1] CRAN (R 4.2.0)
##  readxl    * 1.4.0   2022-03-28 [1] CRAN (R 4.2.0)
##  usethis   * 2.1.6   2022-05-25 [1] CRAN (R 4.2.0)
## 
##  [1] /Library/Frameworks/R.framework/Versions/4.2/Resources/library
## 
## ──────────────────────────────────────────────────────────────────────────────