AMF: Adaptable Weighting Fusion with Multiple Fine-tuning for Image Classification

TL;DR

This paper introduces Adaptable Multi-tuning, a novel fine-tuning framework that dynamically adjusts training strategies for individual samples, improving image classification performance across diverse datasets.

Contribution

The paper proposes a new adaptive fine-tuning method with a policy network to optimize sample-specific training strategies, outperforming standard fine-tuning.

Findings

Outperforms standard fine-tuning by 1.69% on FGVC-Aircraft

Achieves 2.79% improvement on Describable Texture

Maintains comparable performance on Stanford Cars, CIFAR-10, Fashion-MNIST

Abstract

Fine-tuning is widely applied in image classification tasks as a transfer learning approach. It re-uses the knowledge from a source task to learn and obtain a high performance in target tasks. Fine-tuning is able to alleviate the challenge of insufficient training data and expensive labelling of new data. However, standard fine-tuning has limited performance in complex data distributions. To address this issue, we propose the Adaptable Multi-tuning method, which adaptively determines each data sample's fine-tuning strategy. In this framework, multiple fine-tuning settings and one policy network are defined. The policy network in Adaptable Multi-tuning can dynamically adjust to an optimal weighting to feed different samples into models that are trained using different fine-tuning strategies. Our method outperforms the standard fine-tuning approach by 1.69%, 2.79% on the datasets FGVC-Aircraft, and Describable Texture, yielding comparable performance on the datasets Stanford Cars, CIFAR-10, and Fashion-MNIST.