Toward Efficient Convolutional Neural Networks With Structured Ternary Patterns

TL;DR

This paper introduces Structured Ternary Patterns (STePs), a method for designing efficient convolutional neural networks using static, non-learnable filters based on local binary patterns and Haar features, reducing resource demands.

Contribution

It proposes a novel approach to create efficient ConvNet architectures with static filters, significantly decreasing trainable parameters and resource usage without sacrificing accuracy.

Findings

Reduces trainable parameters by 40-80%

Maintains high detection accuracy on multiple datasets

Enables custom STeP-based networks for on-device applications

Abstract

High-efficiency deep learning (DL) models are necessary not only to facilitate their use in devices with limited resources but also to improve resources required for training. Convolutional neural networks (ConvNets) typically exert severe demands on local device resources and this conventionally limits their adoption within mobile and embedded platforms. This brief presents work toward utilizing static convolutional filters generated from the space of local binary patterns (LBPs) and Haar features to design efficient ConvNet architectures. These are referred to as Structured Ternary Patterns (STePs) and can be generated during network initialization in a systematic way instead of having learnable weight parameters thus reducing the total weight updates. The ternary values require significantly less storage and with the appropriate low-level implementation, can also lead to inference improvements. The proposed approach is validated using four image classification datasets, demonstrating that common network backbones can be made more efficient and provide competitive results. It is also demonstrated that it is possible to generate completely custom STeP-based networks that provide good trade-offs for on-device applications such as unmanned aerial vehicle (UAV)-based aerial vehicle detection. The experimental results show that the proposed method maintains high detection accuracy while reducing the trainable parameters by 40-80%. This work motivates further research toward good priors for non-learnable weights that can make DL architectures more efficient without having to alter the network during or after training.