Neural Architecture Search of SPD Manifold Networks

TL;DR

This paper introduces a differentiable neural architecture search method tailored for Symmetric Positive Definite (SPD) manifold networks, resulting in more efficient and lighter models that outperform existing approaches on various recognition tasks.

Contribution

It presents a novel NAS framework for SPD networks, including a new search space and a supernet-based differentiable search algorithm.

Findings

Achieves better accuracy than state-of-the-art SPD networks.

Produces models over three times lighter than previous NAS results.

Demonstrates effectiveness on drone, action, and emotion recognition tasks.

Abstract

In this paper, we propose a new neural architecture search (NAS) problem of Symmetric Positive Definite (SPD) manifold networks, aiming to automate the design of SPD neural architectures. To address this problem, we first introduce a geometrically rich and diverse SPD neural architecture search space for an efficient SPD cell design. Further, we model our new NAS problem with a one-shot training process of a single supernet. Based on the supernet modeling, we exploit a differentiable NAS algorithm on our relaxed continuous search space for SPD neural architecture search. Statistical evaluation of our method on drone, action, and emotion recognition tasks mostly provides better results than the state-of-the-art SPD networks and traditional NAS algorithms. Empirical results show that our algorithm excels in discovering better performing SPD network design and provides models that are more than three times lighter than searched by the state-of-the-art NAS algorithms.