Advancing Supervised Local Learning Beyond Classification with Long-term Feature Bank

TL;DR

This paper introduces FBA, a local learning method with a feature bank that extends beyond classification, conserving memory and matching end-to-end performance in diverse visual tasks.

Contribution

It presents the first successful extension of local learning to multiple visual tasks using a feature bank for better cross-task adaptability.

Findings

FBA reduces GPU memory consumption significantly.

FBA achieves comparable performance to end-to-end methods.

FBA effectively applies beyond classification tasks.

Abstract

Local learning offers an alternative to traditional end-to-end back-propagation in deep neural networks, significantly reducing GPU memory consumption. Although it has shown promise in image classification tasks, its extension to other visual tasks has been limited. This limitation arises primarily from two factors: 1) architectures designed specifically for classification are not readily adaptable to other tasks, which prevents the effective reuse of task-specific knowledge from architectures tailored to different problems; 2) these classification-focused architectures typically lack cross-scale feature communication, leading to degraded performance in tasks like object detection and super-resolution. To address these challenges, we propose the Feature Bank Augmented auxiliary network (FBA), which introduces a simplified design principle and incorporates a feature bank to enhance cross-task adaptability and communication. This work represents the first successful application of local learning methods beyond classification, demonstrating that FBA not only conserves GPU memory but also achieves performance on par with end-to-end approaches across multiple datasets for various visual tasks.