Functional partitioning through competitive learning

TL;DR

This paper introduces a new algorithm that uses competition between models to identify and separate different functional patterns in datasets, improving performance on regression tasks.

Contribution

A novel partitioning algorithm based on competitive learning that enables model specialization and improves regression performance.

Findings

The algorithm successfully detects and separates distinct functional patterns in datasets.

Modular models using the partitioning algorithm achieved up to 56% loss reduction compared to single models.

The method was validated on mechanical stress and strain data in porous structures.

Abstract

Datasets often incorporate various functional patterns related to different aspects or regimes, which are typically not equally present throughout the dataset. We propose a novel partitioning algorithm that utilizes competition between models to detect and separate these functional patterns. This competition is induced by multiple models iteratively submitting their predictions for the dataset, with the best prediction for each data point being rewarded with training on that data point. This reward mechanism amplifies each model's strengths and encourages specialization in different patterns. The specializations can then be translated into a partitioning scheme. We validate our concept with datasets with clearly distinct functional patterns, such as mechanical stress and strain data in a porous structure. Our partitioning algorithm produces valuable insights into the datasets' structure, which can serve various further applications. As a demonstration of one exemplary usage, we set up modular models consisting of multiple expert models, each learning a single partition, and compare their performance on more than twenty popular regression problems with single models learning all partitions simultaneously. Our results show significant improvements, with up to 56% loss reduction, confirming our algorithm's utility.

Figures