Networked Information Aggregation for Binary Classification

TL;DR

This paper analyzes how a sequential, networked logistic regression protocol on a DAG can aggregate information, identifying depth as a key bottleneck for achieving low excess loss.

Contribution

It extends previous linear regression results to classification, providing bounds on excess loss and highlighting the impact of network depth on information aggregation.

Findings

Excess loss is bounded by $O(M/\sqrt{D})$ under certain feature observation conditions.

Instances exist with excess loss at least $\Omega(k/D)$, showing a fundamental bottleneck.

Network depth critically limits the effectiveness of distributed logistic regression.

Abstract

We study networked binary classification on a directed acyclic graph (DAG) where each agent observes only a subset of the feature columns of a shared dataset. Agents act sequentially along the DAG: each receives prediction columns from its parents (if any), augments its local features with these columns, fits a logistic predictor by minimizing binary cross-entropy (BCE), and forwards its prediction column to its outgoing neighbors. We ask whether this sequential distributed training procedure achieves information aggregation, meaning that some agent attains small excess loss compared to the best logistic predictor trained with access to all feature columns. This question was studied for linear regression under squared loss by Kearns, Roth, and Ryu (SODA 2026). Extending their guarantees to classification is nontrivial because their analysis relies on quadratic structure that does not directly transfer to BCE with a logistic link. We analyze the resulting sequential logit-passing protocol and prove: (i) an excess loss upper bound of $O(M/\sqrt{D})$ on depth-$D$ paths under the condition that every $M$ contiguous subsequence of $M$ agents collectively observe all features, and (ii) a close lower bound showing instances with excess loss of at least $\Omega(k/D)$ where $k$ is the dimension of the feature space. Together, these results identify network depth as a fundamental bottleneck for information aggregation in networked logistic regression.