Adaptive Data Selection for Multi-Layer Perceptron Training: A Sub-linear Value-Driven Method

TL;DR

This paper introduces DVC, a novel adaptive data selection method for training multi-layer perceptrons that considers hierarchical data contributions and dynamically evolves during training, outperforming existing methods.

Contribution

The paper presents DVC, a new budget-aware data selection approach that decomposes data value into layer-wise and global contributions, addressing scalability and nonlinear transformation challenges.

Findings

DVC outperforms existing methods in accuracy and F1 scores across six datasets.

The approach effectively balances exploration and exploitation with UCB.

Hierarchical data evaluation improves training efficiency and model performance.

Abstract

Data selection is one of the fundamental problems in neural network training, particularly for multi-layer perceptrons (MLPs) where identifying the most valuable training samples from massive, multi-source, and heterogeneous data sources under budget constraints poses significant challenges. Existing data selection methods, including coreset construction, data Shapley values, and influence functions, suffer from critical limitations: they oversimplify nonlinear transformations, ignore informative intermediate representations in hidden layers, or fail to scale to larger MLPs due to high computational complexity. In response, we propose DVC (Data Value Contribution), a novel budget-aware method for evaluating and selecting data for MLP training that accounts for the dynamic evolution of network parameters during training. The DVC method decomposes data contribution into Layer Value Contribution (LVC) and Global Value Contribution (GVC), employing six carefully designed metrics and corresponding efficient algorithms to capture data characteristics across three dimensions--quality, relevance, and distributional diversity--at different granularities. DVC integrates these assessments with an Upper Confidence Bound (UCB) algorithm for adaptive source selection that balances exploration and exploitation. Extensive experiments across six datasets and eight baselines demonstrate that our method consistently outperforms existing approaches under various budget constraints, achieving superior accuracy and F1 scores. Our approach represents the first systematic treatment of hierarchical data evaluation for neural networks, providing both theoretical guarantees and practical advantages for large-scale machine learning systems.