FixyNN: Efficient Hardware for Mobile Computer Vision via Transfer Learning

TL;DR

FixyNN introduces a hybrid CNN hardware architecture combining a fixed feature extractor with a programmable accelerator, significantly improving energy efficiency for mobile vision tasks while maintaining high accuracy.

Contribution

The paper presents FixyNN, a novel hardware design that leverages transfer learning to enhance energy efficiency in mobile CNN inference by combining fixed and programmable components.

Findings

Achieves up to 26.6 TOPS/W energy efficiency.

Maintains <1% accuracy loss across six datasets.

Nearly doubles efficiency compared to conventional accelerators.

Abstract

The computational demands of computer vision tasks based on state-of-the-art Convolutional Neural Network (CNN) image classification far exceed the energy budgets of mobile devices. This paper proposes FixyNN, which consists of a fixed-weight feature extractor that generates ubiquitous CNN features, and a conventional programmable CNN accelerator which processes a dataset-specific CNN. Image classification models for FixyNN are trained end-to-end via transfer learning, with the common feature extractor representing the transfered part, and the programmable part being learnt on the target dataset. Experimental results demonstrate FixyNN hardware can achieve very high energy efficiencies up to 26.6 TOPS/W ($4.81 \times$ better than iso-area programmable accelerator). Over a suite of six datasets we trained models via transfer learning with an accuracy loss of $<1\%$ resulting in up to 11.2 TOPS/W - nearly $2 \times$ more efficient than a conventional programmable CNN accelerator of the same area.