Hierarchical Consensus Network for Multiview Feature Learning

TL;DR

This paper introduces a hierarchical consensus network (HCN) that effectively integrates multiview features by capturing class-level, instance-level, and global consensus, leading to improved discriminative feature learning.

Contribution

The paper proposes a novel HCN model that leverages hierarchical consensus indices inspired by CCA and contrastive learning to enhance multiview feature integration.

Findings

HCN outperforms state-of-the-art methods on four multiview datasets.

Hierarchical consensus improves feature discriminability.

The method effectively captures class, instance, and global consensus.

Abstract

Multiview feature learning aims to learn discriminative features by integrating the distinct information in each view. However, most existing methods still face significant challenges in learning view-consistency features, which are crucial for effective multiview learning. Motivated by the theories of CCA and contrastive learning in multiview feature learning, we propose the hierarchical consensus network (HCN) in this paper. The HCN derives three consensus indices for capturing the hierarchical consensus across views, which are classifying consensus, coding consensus, and global consensus, respectively. Specifically, classifying consensus reinforces class-level correspondence between views from a CCA perspective, while coding consensus closely resembles contrastive learning and reflects contrastive comparison of individual instances. Global consensus aims to extract consensus information from two perspectives simultaneously. By enforcing the hierarchical consensus, the information within each view is better integrated to obtain more comprehensive and discriminative features. The extensive experimental results obtained on four multiview datasets demonstrate that the proposed method significantly outperforms several state-of-the-art methods.