Multi-Objective Optimisation of Multi-Output Neural Trees

TL;DR

This paper introduces a multi-output neural tree (MONT) algorithm optimized via NSGA-III for classification, demonstrating superior accuracy over traditional classifiers on benchmark problems.

Contribution

The paper presents a novel multi-output neural tree method trained with NSGA-III, addressing multi-objective optimization in classification tasks.

Findings

MONT achieves high classification accuracy on benchmark datasets.

NSGA-III outperforms other evolutionary algorithms in Pareto optimization.

MONT outperforms traditional classifiers like SVM and decision trees.

Abstract

We propose an algorithm and a new method to tackle the classification problems. We propose a multi-output neural tree (MONT) algorithm, which is an evolutionary learning algorithm trained by the non-dominated sorting genetic algorithm (NSGA)-III. Since evolutionary learning is stochastic, a hypothesis found in the form of MONT is unique for each run of evolutionary learning, i.e., each hypothesis (tree) generated bears distinct properties compared to any other hypothesis both in topological space and parameter-space. This leads to a challenging optimisation problem where the aim is to minimise the tree-size and maximise the classification accuracy. Therefore, the Pareto-optimality concerns were met by hypervolume indicator analysis. We used nine benchmark classification learning problems to evaluate the performance of the MONT. As a result of our experiments, we obtained MONTs which are able to tackle the classification problems with high accuracy. The performance of MONT emerged better over a set of problems tackled in this study compared with a set of well-known classifiers: multilayer perceptron, reduced-error pruning tree, naive Bayes classifier, decision tree, and support vector machine. Moreover, the performances of three versions of MONT's training using genetic programming, NSGA-II, and NSGA-III suggest that the NSGA-III gives the best Pareto-optimal solution.