Problem Evolution: A new approach to problem solving systems

TL;DR

This paper introduces a novel method called problem evolution that evaluates problem solving systems by finding the most complex problem they can solve, demonstrated on neural networks with linguistic complexity.

Contribution

It proposes a new approach to evaluate systems by evolving problems rather than solving them, applied to neural networks and pattern recognition.

Findings

Successfully ranked neural network configurations by their pattern recognition ability.

Demonstrated the effectiveness of the problem evolution approach in system evaluation.

Showed that linguistic complexity can quantify problem difficulty.

Abstract

In this paper we present a novel tool to evaluate problem solving systems. Instead of using a system to solve a problem, we suggest using the problem to evaluate the system. By finding a numerical representation of a problem's complexity, one can implement genetic algorithm to search for the most complex problem the given system can solve. This allows a comparison between different systems that solve the same set of problems. In this paper we implement this approach on pattern recognition neural networks to try and find the most complex pattern a given configuration can solve. The complexity of the pattern is calculated using linguistic complexity. The results demonstrate the power of the problem evolution approach in ranking different neural network configurations according to their pattern recognition abilities. Future research and implementations of this technique are also discussed.