Improving Generalization Ability of Genetic Programming: Comparative Study

TL;DR

This paper reviews techniques for controlling bloat in Genetic Programming, tests four methods on various problems, and proposes a combined approach that improves generalization and solution quality.

Contribution

It provides a comprehensive classification of bloat control techniques and demonstrates that combining double tournament with Tarpeian method enhances GP performance.

Findings

Combined method outperforms individual techniques.

Bloat control improves generalization in GP.

Different methods perform variably across problems.

Abstract

In the field of empirical modeling using Genetic Programming (GP), it is important to evolve solution with good generalization ability. Generalization ability of GP solutions get affected by two important issues: bloat and over-fitting. Bloat is uncontrolled growth of code without any gain in fitness and important issue in GP. We surveyed and classified existing literature related to different techniques used by GP research community to deal with the issue of bloat. Moreover, the classifications of different bloat control approaches and measures for bloat are discussed. Next, we tested four bloat control methods: Tarpeian, double tournament, lexicographic parsimony pressure with direct bucketing and ratio bucketing on six different problems and identified where each bloat control method performs well on per problem basis. Based on the analysis of each method, we combined two methods: double tournament (selection method) and Tarpeian method (works before evaluation) to avoid bloated solutions and compared with the results obtained from individual performance of double tournament method. It was found that the results were improved with this combination of two methods.