# Comparing raw score difference, multilevel modeling, and structural equation modeling methods for estimating discrepancy in dyads

## TL;DR

This study compares three methods for estimating differences in paired data and finds that raw score difference and structural equation modeling are more reliable than multilevel modeling.

## Contribution

The study provides empirical evidence on the reliability of discrepancy estimation methods in dyadic data analysis.

## Key findings

- MLM discrepancy estimates had poor reliability, especially with high ICC and low cluster numbers.
- RSD and SEM methods showed similar performance and were not affected by design factors.
- RSD and SEM are recommended for practical use in estimating dyadic discrepancy.

## Abstract

Dyadic data analysis is commonly used in psychological research involving pairs of individuals in a nested relationship, such as parent and child, student and teacher, and pairs of spouses. There are several methods for calculating dyadic discrepancy (i.e., difference) scores, and purpose of the present study was to explore which of these methods produced the most accurate discrepancy estimates and most accurate outcome prediction.

Using a Monte Carlo simulation, the present study compared three methods for estimating discrepancy scores in dyad pairs: raw score difference (RSD), empirical Bayes estimates from multilevel modeling (MLM), and factor scores from structural equation modeling (SEM). Design factors for this simulation included intraclass correlation (ICC), cluster number, reliability estimates, effect size of discrepancy, and effect size variance.

Results suggest discrepancy estimates from MLM had poor reliability compared to RSD and SEM methods. These findings were driven primarily by having a high ICC, high effect size variance, and low number of clusters. None of the design factors had an appreciable impact on either the RSD or SEM estimates.

RSD and SEM methods performed similarly, and are recommended for practical use in estimating discrepancy values. MLM is not recommended as it featured comparatively poor reliability.