Hierarchical Feature-level Reverse Propagation for Post-Training Neural Networks

TL;DR

This paper introduces a hierarchical, decoupled post-training framework for neural networks that reconstructs feature maps from labels, enabling interpretable, flexible, and efficient training beyond traditional backpropagation.

Contribution

It formalizes feature-level reverse computation as well-posed optimization problems, pioneering a new training paradigm extending gradient backpropagation to feature backpropagation.

Findings

Achieves superior generalization on image classification benchmarks.

Improves computational efficiency over traditional training methods.

Provides interpretable insights into network mechanisms.

Abstract

End-to-end autonomous driving has emerged as a dominant paradigm, yet its highly entangled black-box models pose significant challenges in terms of interpretability and safety assurance. To improve model transparency and training flexibility, this paper proposes a hierarchical and decoupled post-training framework tailored for pretrained neural networks. By reconstructing intermediate feature maps from ground-truth labels, surrogate supervisory signals are introduced at transitional layers to enable independent training of specific components, thereby avoiding the complexity and coupling of conventional end-to-end backpropagation and providing interpretable insights into networks' internal mechanisms. To the best of our knowledge, this is the first method to formalize feature-level reverse computation as well-posed optimization problems, which we rigorously reformulate as systems of linear equations or least squares problems. This establishes a novel and efficient training paradigm that extends gradient backpropagation to feature backpropagation. Extensive experiments on multiple standard image classification benchmarks demonstrate that the proposed method achieves superior generalization performance and computational efficiency compared to traditional training approaches, validating its effectiveness and potential.