An Online Updating Approach for Estimating and Testing Mediation Effects with Big Data Streams
Xueyan Bai, Haixiang Zhang

TL;DR
This paper introduces an online updating method for efficient estimation and testing of mediation effects in large-scale streaming data, improving computational speed for various mediation tests.
Contribution
It proposes a novel online updating algorithm that enhances computational efficiency for mediation analysis in big data streams, applicable to linear and logistic models.
Findings
Significant speed improvements in mediation tests.
Effective application demonstrated on real-world data.
Robust performance shown through extensive simulations.
Abstract
The use of mediation analysis has become increasingly popular in various research fields in recent years. The primary objective of mediation analysis is to examine the indirect effects along the pathways from exposure to outcome. Meanwhile, the advent of data collection technology has sparked a surge of interest in the burgeoning field of big data analysis, where mediation analysis of streaming data sets has recently garnered significant attention. The enormity of the data, however, results in an augmented computational burden. The present study proposes an online updating approach to address this issue, aiming to estimate and test mediation effects in the context of linear and logistic mediation models with massive data streams. The proposed algorithm significantly enhances the computational efficiency of Sobel test, adjusted Sobel test, joint significance test, and adjusted joint…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsBig Data and Business Intelligence · Data Stream Mining Techniques · Privacy-Preserving Technologies in Data
