Language Inference with Multi-head Automata through Reinforcement Learning

TL;DR

This paper explores using reinforcement learning to train multi-head automata for recognizing formal languages, comparing Q-learning and genetic algorithms, with genetic algorithms generally performing better.

Contribution

It introduces a novel model of multi-head automata optimized via reinforcement learning, applying two algorithms to recognize formal languages.

Findings

Genetic algorithms outperform Q-learning in overall accuracy.

Q-learning finds solutions faster for regular languages.

Reinforcement learning effectively models language recognition with automata.

Abstract

The purpose of this paper is to use reinforcement learning to model learning agents which can recognize formal languages. Agents are modeled as simple multi-head automaton, a new model of finite automaton that uses multiple heads, and six different languages are formulated as reinforcement learning problems. Two different algorithms are used for optimization. First algorithm is Q-learning which trains gated recurrent units to learn optimal policies. The second one is genetic algorithm which searches for the optimal solution by using evolution inspired operations. The results show that genetic algorithm performs better than Q-learning algorithm in general but Q-learning algorithm finds solutions faster for regular languages.