Prespecification of Structure for Optimizing Data Collection and Research Transparency by Leveraging Conditional Independencies

TL;DR

This paper proposes a method to optimize data collection by using structural causal models and conditional independencies, reducing participant burden and increasing research transparency.

Contribution

It introduces a novel approach leveraging causal structures to identify essential variables, streamlining data collection without compromising analysis validity.

Findings

Simulation results show minimal impact on analysis accuracy.

Method improves resource allocation in data collection.

Enhances transparency and reliability of research findings.

Abstract

Data collection and research methodology represents a critical part of the research pipeline. On the one hand, it is important that we collect data in a way that maximises the validity of what we are measuring, which may involve the use of long scales with many items. On the other hand, collecting a large number of items across multiple scales results in participant fatigue, and expensive and time consuming data collection. It is therefore important that we use the available resources optimally. In this work, we consider how a consideration for theory and the associated causal/structural model can help us to streamline data collection procedures by not wasting time collecting data for variables which are not causally critical for subsequent analysis. This not only saves time and enables us to redirect resources to attend to other variables which are more important, but also increases research transparency and the reliability of theory testing. In order to achieve this streamlined data collection, we leverage structural models, and Markov conditional independency structures implicit in these models to identify the substructures which are critical for answering a particular research question. In this work, we review the relevant concepts and present a number of didactic examples with the hope that psychologists can use these techniques to streamline their data collection process without invalidating the subsequent analysis. We provide a number of simulation results to demonstrate the limited analytical impact of this streamlining.