Transferability of Convolutional Neural Networks in Stationary Learning Tasks

TL;DR

This paper demonstrates that convolutional neural networks trained on small stationary signal windows can effectively scale to larger problems, reducing computational costs while maintaining performance, supported by theoretical bounds and experimental validation.

Contribution

The paper introduces a framework showing CNNs trained on small windows can generalize to larger scales in stationary tasks, with theoretical performance bounds and practical applications.

Findings

CNN trained on small windows performs nearly as well on larger windows

Theoretical bounds on performance degradation are established

CNNs can handle large-scale multi-agent problems efficiently

Abstract

Recent advances in hardware and big data acquisition have accelerated the development of deep learning techniques. For an extended period of time, increasing the model complexity has led to performance improvements for various tasks. However, this trend is becoming unsustainable and there is a need for alternative, computationally lighter methods. In this paper, we introduce a novel framework for efficient training of convolutional neural networks (CNNs) for large-scale spatial problems. To accomplish this we investigate the properties of CNNs for tasks where the underlying signals are stationary. We show that a CNN trained on small windows of such signals achieves a nearly performance on much larger windows without retraining. This claim is supported by our theoretical analysis, which provides a bound on the performance degradation. Additionally, we conduct thorough experimental analysis on two tasks: multi-target tracking and mobile infrastructure on demand. Our results show that the CNN is able to tackle problems with many hundreds of agents after being trained with fewer than ten. Thus, CNN architectures provide solutions to these problems at previously computationally intractable scales.