RRR-Net: Reusing, Reducing, and Recycling a Deep Backbone Network

TL;DR

This paper proposes techniques to reuse and compress large pre-trained backbone networks like ResNet152, creating smaller, faster models that maintain or improve performance across diverse image classification tasks.

Contribution

It introduces methods to reduce, split, and ensemble pre-trained backbones, achieving significant size and speed improvements without sacrificing accuracy.

Findings

Reduced ResNet152 from 51 to 5 blocks with minimal performance loss.

Created an ensemble of sub-networks with the same parameters and FLOPs.

Achieved comparable or better performance with smaller, faster models.

Abstract

It has become mainstream in computer vision and other machine learning domains to reuse backbone networks pre-trained on large datasets as preprocessors. Typically, the last layer is replaced by a shallow learning machine of sorts; the newly-added classification head and (optionally) deeper layers are fine-tuned on a new task. Due to its strong performance and simplicity, a common pre-trained backbone network is ResNet152.However, ResNet152 is relatively large and induces inference latency. In many cases, a compact and efficient backbone with similar performance would be preferable over a larger, slower one. This paper investigates techniques to reuse a pre-trained backbone with the objective of creating a smaller and faster model. Starting from a large ResNet152 backbone pre-trained on ImageNet, we first reduce it from 51 blocks to 5 blocks, reducing its number of parameters and FLOPs by more than 6 times, without significant performance degradation. Then, we split the model after 3 blocks into several branches, while preserving the same number of parameters and FLOPs, to create an ensemble of sub-networks to improve performance. Our experiments on a large benchmark of $40$ image classification datasets from various domains suggest that our techniques match the performance (if not better) of ``classical backbone fine-tuning'' while achieving a smaller model size and faster inference speed.