Deep Learning-Based Estimation and Goodness-of-Fit for Large-Scale Confirmatory Item Factor Analysis

TL;DR

This paper introduces deep learning methods for parameter estimation and goodness-of-fit testing in large-scale confirmatory item factor analysis, improving efficiency and accuracy over existing techniques.

Contribution

It extends a deep learning algorithm to the confirmatory IFA setting with constraints and develops novel simulation-based GOF tests, including a relative fit index.

Findings

The deep learning estimator performs comparably to state-of-the-art methods in less time.

C2ST-based GOF tests effectively control type I error and detect misspecification.

The RFI's sampling distribution varies with sample size.

Abstract

We investigate novel parameter estimation and goodness-of-fit (GOF) assessment methods for large-scale confirmatory item factor analysis (IFA) with many respondents, items, and latent factors. For parameter estimation, we extend Urban and Bauer's (2021) deep learning algorithm for exploratory IFA to the confirmatory setting by showing how to handle constraints on loadings and factor correlations. For GOF assessment, we explore simulation-based tests and indices that extend the classifier two-sample test (C2ST), a method that tests whether a deep neural network can distinguish between observed data and synthetic data sampled from a fitted IFA model. Proposed extensions include a test of approximate fit wherein the user specifies what percentage of observed and synthetic data should be distinguishable as well as a relative fit index (RFI) that is similar in spirit to the RFIs used in structural equation modeling. Via simulation studies, we show that: (1) the confirmatory extension of Urban and Bauer's (2021) algorithm obtains comparable estimates to a state-of-the-art estimation procedure in less time; (2) C2ST-based GOF tests control the empirical type I error rate and detect when the latent dimensionality is misspecified; and (3) the sampling distribution of the C2ST-based RFI depends on the sample size.