Learning Solving Procedure for Artificial Neural Network

TL;DR

This paper introduces a novel learning paradigm called Learning Solving Procedure (LSP) that enables neural networks to learn complex algorithms and reasoning processes, advancing neural-symbolic integration.

Contribution

The paper proposes the LSP framework that learns algorithms as sequences of tasks, demonstrating its effectiveness on complex problems like sorting and Hanoi Tower.

Findings

LSP effectively learns algorithms for complex problems.

LSP demonstrates scalability and efficiency.

LSP validates complex reasoning capabilities.

Abstract

It is expected that progress toward true artificial intelligence will be achieved through the emergence of a system that integrates representation learning and complex reasoning (LeCun et al. 2015). In response to this prediction, research has been conducted on implementing the symbolic reasoning of a von Neumann computer in an artificial neural network (Graves et al. 2016; Graves et al. 2014; Reed et al. 2015). However, these studies have many limitations in realizing neural-symbolic integration (Jaeger. 2016). Here, we present a new learning paradigm: a learning solving procedure (LSP) that learns the procedure for solving complex problems. This is not accomplished merely by learning input-output data, but by learning algorithms through a solving procedure that obtains the output as a sequence of tasks for a given input problem. The LSP neural network system not only learns simple problems of addition and multiplication, but also the algorithms of complicated problems, such as complex arithmetic expression, sorting, and Hanoi Tower. To realize this, the LSP neural network structure consists of a deep neural network and long short-term memory, which are recursively combined. Through experimentation, we demonstrate the efficiency and scalability of LSP and its validity as a mechanism of complex reasoning.