Double Relief with progressive weighting function

TL;DR

This paper introduces an improved version of the Relief feature weighting algorithm that enhances robustness to data-specific characteristics, validated through experimental testing.

Contribution

It presents a refined extension of Relief with a progressive weighting function, increasing robustness and performance in feature relevance estimation.

Findings

Improved robustness of the Relief extension demonstrated

Experimental results show increased accuracy in feature weights

The new method adapts better to data characteristics

Abstract

Feature weighting algorithms try to solve a problem of great importance nowadays in machine learning: The search of a relevance measure for the features of a given domain. This relevance is primarily used for feature selection as feature weighting can be seen as a generalization of it, but it is also useful to better understand a problem's domain or to guide an inductor in its learning process. Relief family of algorithms are proven to be very effective in this task. On previous work, a new extension was proposed that aimed for improving the algorithm's performance and it was shown that in certain cases it improved the weights' estimation accuracy. However, it also seemed to be sensible to some characteristics of the data. An improvement of that previously presented extension is presented in this work that aims to make it more robust to problem specific characteristics. An experimental design is proposed to test its performance. Results of the tests prove that it indeed increase the robustness of the previously proposed extension.