Minimal Learning Machine for Multi-Label Learning

TL;DR

This paper introduces a simple, deterministic multi-label learning method based on the minimal learning machine that achieves state-of-the-art results and allows for uncertainty estimation without cross-validation.

Contribution

It adapts the minimal learning machine to multi-label learning using distance mapping and inverse distance weighting, with a novel hyper-parameter selection method.

Findings

Achieves state-of-the-art performance on small to moderate multi-label datasets.

Fully deterministic hyper-parameter tuning via a closed-form ranking loss statistic.

Enables uncertainty estimation in multi-label predictions.

Abstract

Distance-based supervised method, the minimal learning machine, constructs a predictive model from data by learning a mapping between input and output distance matrices. In this paper, we propose new methods and evaluate how their core component, the distance mapping, can be adapted to multi-label learning. The proposed approach is based on combining the distance mapping with an inverse distance weighting. Although the proposal is one of the simplest methods in the multi-label learning literature, it achieves state-of-the-art performance for small to moderate-sized multi-label learning problems. In addition to its simplicity, the proposed method is fully deterministic: Its hyper-parameter can be selected via ranking loss-based statistic which has a closed form, thus avoiding conventional cross-validation-based hyper-parameter tuning. In addition, due to its simple linear distance mapping-based construction, we demonstrate that the proposed method can assess the uncertainty of the predictions for multi-label classification, which is a valuable capability for data-centric machine learning pipelines.