Self-Supervised Neural Architecture Search for Multimodal Deep Neural Networks

TL;DR

This paper introduces a self-supervised learning approach to neural architecture search for multimodal deep neural networks, enabling architecture design from unlabeled data and reducing reliance on labeled datasets.

Contribution

It presents a novel SSL-based NAS method specifically tailored for multimodal DNNs, integrating architecture search and pretraining without labeled data.

Findings

Successfully designed architectures from unlabeled data

Improved performance in multimodal DNNs

Reduces dependence on labeled training data

Abstract

Neural architecture search (NAS), which automates the architectural design process of deep neural networks (DNN), has attracted increasing attention. Multimodal DNNs that necessitate feature fusion from multiple modalities benefit from NAS due to their structural complexity; however, constructing an architecture for multimodal DNNs through NAS requires a substantial amount of labeled training data. Thus, this paper proposes a self-supervised learning (SSL) method for architecture search of multimodal DNNs. The proposed method applies SSL comprehensively for both the architecture search and model pretraining processes. Experimental results demonstrated that the proposed method successfully designed architectures for DNNs from unlabeled training data.