# Ensemble of heterogeneous flexible neural trees using multiobjective genetic programming

## TL;DR

This paper introduces a multiobjective genetic programming approach to create heterogeneous neural trees that optimize accuracy and simplicity, forming an ensemble system that outperforms existing methods across various datasets.

## Contribution

It proposes a novel multiobjective genetic programming method for evolving heterogeneous neural trees and forming ensembles, enhancing model diversity and performance.

## Key findings

- The ensemble system outperforms other prediction methods on multiple datasets.
- Heterogeneous neural trees improve data insight during learning.
- The approach effectively balances accuracy and complexity.

## Abstract

Machine learning algorithms are inherently multiobjective in nature, where approximation error minimization and model's complexity simplification are two conflicting objectives. We proposed a multiobjective genetic programming (MOGP) for creating a heterogeneous flexible neural tree (HFNT), tree-like flexible feedforward neural network model. The functional heterogeneity in neural tree nodes was introduced to capture a better insight of data during learning because each input in a dataset possess different features. MOGP guided an initial HFNT population towards Pareto-optimal solutions, where the final population was used for making an ensemble system. A diversity index measure along with approximation error and complexity was introduced to maintain diversity among the candidates in the population. Hence, the ensemble was created by using accurate, structurally simple, and diverse candidates from MOGP final population. Differential evolution algorithm was applied to fine-tune the underlying parameters of the selected candidates. A comprehensive test over classification, regression, and time-series datasets proved the efficiency of the proposed algorithm over other available prediction methods. Moreover, the heterogeneous creation of HFNT proved to be efficient in making ensemble system from the final population.