On the performance of deep learning for numerical optimization: an application to protein structure prediction

TL;DR

This paper evaluates the effectiveness of deep learning models, optimized via neural architecture search, for solving global optimization problems including protein structure prediction, demonstrating competitive results with traditional methods.

Contribution

It introduces a NAS-based approach to generate lightweight neural networks tailored for global optimization tasks like protein structure prediction.

Findings

NAS-generated models achieve competitive results

Lightweight architectures outperform large networks in efficiency

Effective for complex optimization problems

Abstract

Deep neural networks have recently drawn considerable attention to build and evaluate artificial learning models for perceptual tasks. Here, we present a study on the performance of the deep learning models to deal with global optimization problems. The proposed approach adopts the idea of the neural architecture search (NAS) to generate efficient neural networks for solving the problem at hand. The space of network architectures is represented using a directed acyclic graph and the goal is to find the best architecture to optimize the objective function for a new, previously unknown task. Different from proposing very large networks with GPU computational burden and long training time, we focus on searching for lightweight implementations to find the best architecture. The performance of NAS is first analyzed through empirical experiments on CEC 2017 benchmark suite. Thereafter, it is applied to a set of protein structure prediction (PSP) problems. The experiments reveal that the generated learning models can achieve competitive results when compared to hand-designed algorithms; given enough computational budget