Evolving Multi-Channel Confidence-Aware Activation Functions for Missing Data with Channel Propagation

TL;DR

This paper introduces 3C-EA, a genetic programming evolved multi-channel activation function that incorporates missingness and confidence information, along with ChannelProp for propagating these signals through neural networks, improving performance with missing data.

Contribution

It presents a novel multivariate activation function and a propagation algorithm to handle missing data more effectively in neural networks.

Findings

3C-EA improves classification accuracy with missing data.

ChannelProp maintains reliability signals across layers.

The approach outperforms traditional methods on datasets with various missingness types.

Abstract

Learning in the presence of missing data can result in biased predictions and poor generalizability, among other difficulties, which data imputation methods only partially address. In neural networks, activation functions significantly affect performance yet typical options (e.g., ReLU, Swish) operate only on feature values and do not account for missingness indicators or confidence scores. We propose Three-Channel Evolved Activations (3C-EA), which we evolve using Genetic Programming to produce multivariate activation functions f(x, m, c) in the form of trees that take (i) the feature value x, (ii) a missingness indicator m, and (iii) an imputation confidence score c. To make these activations useful beyond the input layer, we introduce ChannelProp, an algorithm that deterministically propagates missingness and confidence values via linear layers based on weight magnitudes, retaining reliability signals throughout the network. We evaluate 3C-EA and ChannelProp on datasets with natural and injected (MCAR/MAR/MNAR) missingness at multiple rates under identical preprocessing and splits. Results indicate that integrating missingness and confidence inputs into the activation search improves classification performance under missingness.