GroSS: Group-Size Series Decomposition for Grouped Architecture Search

TL;DR

GroSS introduces a differentiable tensor factorization method that enables simultaneous training of various grouped convolution configurations within neural networks, streamlining architecture search.

Contribution

It is the first method to allow concurrent training of different group sizes and combinations in grouped convolutions, facilitating efficient architecture search.

Findings

Enables training of multiple group configurations simultaneously

Improves efficiency of grouped convolution architecture search

Demonstrates effectiveness on multiple datasets and networks

Abstract

We present a novel approach which is able to explore the configuration of grouped convolutions within neural networks. Group-size Series (GroSS) decomposition is a mathematical formulation of tensor factorisation into a series of approximations of increasing rank terms. GroSS allows for dynamic and differentiable selection of factorisation rank, which is analogous to a grouped convolution. Therefore, to the best of our knowledge, GroSS is the first method to enable simultaneous training of differing numbers of groups within a single layer, as well as all possible combinations between layers. In doing so, GroSS is able to train an entire grouped convolution architecture search-space concurrently. We demonstrate this through architecture searches with performance objectives on multiple datasets and networks. GroSS enables more effective and efficient search for grouped convolutional architectures.