Informed deep hierarchical classification: a non-standard analysis inspired approach

TL;DR

This paper introduces LH-DNN, a novel deep neural network architecture for hierarchical classification that leverages non-standard analysis and multi-objective optimization, achieving efficient and effective classification without complex loss weighting.

Contribution

It presents a new architecture called LH-DNN that integrates projection operators for hierarchical classification, inspired by non-standard analysis and multi-objective optimization, outperforming existing methods in efficiency.

Findings

Achieves comparable or better accuracy than B-CNN on benchmarks.

Reduces training parameters, epochs, and computational time.

Effectively learns hierarchical relations without ad-hoc loss weighting.

Abstract

This work proposes a novel approach to the deep hierarchical classification task, i.e., the problem of classifying data according to multiple labels organized in a rigid parent-child structure. It consists in a multi-output deep neural network equipped with specific projection operators placed before each output layer. The design of such an architecture, called lexicographic hybrid deep neural network (LH-DNN), has been possible by combining tools from different and quite distant research fields: lexicographic multi-objective optimization, non-standard analysis, and deep learning. To assess the efficacy of the approach, the resulting network is compared against the B-CNN, a convolutional neural network tailored for hierarchical classification tasks, on the CIFAR10, CIFAR100 (where it has been originally and recently proposed before being adopted and tuned for multiple real-world applications) and Fashion-MNIST benchmarks. Evidence states that an LH-DNN can achieve comparable if not superior performance, especially in the learning of the hierarchical relations, in the face of a drastic reduction of the learning parameters, training epochs, and computational time, without the need for ad-hoc loss functions weighting values.