Transfer learning optimization based on evolutionary selective fine tuning

TL;DR

BioTune is an evolutionary adaptive fine-tuning method that selectively updates model layers to improve transfer learning efficiency and accuracy across diverse image classification tasks.

Contribution

The paper introduces BioTune, a novel evolutionary algorithm-based approach for selective layer fine-tuning, reducing computational costs and enhancing transfer learning performance.

Findings

BioTune outperforms existing methods like AutoRGN and LoRA in accuracy.

BioTune reduces the number of trainable parameters during fine-tuning.

BioTune demonstrates efficiency gains across nine diverse image datasets.

Abstract

Deep learning has shown substantial progress in image analysis. However, the computational demands of large, fully trained models remain a consideration. Transfer learning offers a strategy for adapting pre-trained models to new tasks. Traditional fine-tuning often involves updating all model parameters, which can potentially lead to overfitting and higher computational costs. This paper introduces BioTune, an evolutionary adaptive fine-tuning technique that selectively fine-tunes layers to enhance transfer learning efficiency. BioTune employs an evolutionary algorithm to identify a focused set of layers for fine-tuning, aiming to optimize model performance on a given target task. Evaluation across nine image classification datasets from various domains indicates that BioTune achieves competitive or improved accuracy and efficiency compared to existing fine-tuning methods such as AutoRGN and LoRA. By concentrating the fine-tuning process on a subset of relevant layers, BioTune reduces the number of trainable parameters, potentially leading to decreased computational cost and facilitating more efficient transfer learning across diverse data characteristics and distributions.