OpenViewer: Openness-Aware Multi-View Learning

TL;DR

OpenViewer is a novel multi-view learning framework that enhances interpretability and generalization in open environments by simulating unknown samples, transparent data integration, and open-set training.

Contribution

It introduces a comprehensive openness-aware framework with a pseudo-unknown sample generator, an expression-enhanced deep unfolding network, and an open-set training regime, addressing interpretability and generalization challenges.

Findings

Effectively handles unknown samples in multi-view learning

Improves interpretability through functional prior modules

Enhances recognition accuracy for both known and unknown data

Abstract

Multi-view learning methods leverage multiple data sources to enhance perception by mining correlations across views, typically relying on predefined categories. However, deploying these models in real-world scenarios presents two primary openness challenges. 1) Lack of Interpretability: The integration mechanisms of multi-view data in existing black-box models remain poorly explained; 2) Insufficient Generalization: Most models are not adapted to multi-view scenarios involving unknown categories. To address these challenges, we propose OpenViewer, an openness-aware multi-view learning framework with theoretical support. This framework begins with a Pseudo-Unknown Sample Generation Mechanism to efficiently simulate open multi-view environments and previously adapt to potential unknown samples. Subsequently, we introduce an Expression-Enhanced Deep Unfolding Network to intuitively promote interpretability by systematically constructing functional prior-mapping modules and effectively providing a more transparent integration mechanism for multi-view data. Additionally, we establish a Perception-Augmented Open-Set Training Regime to significantly enhance generalization by precisely boosting confidences for known categories and carefully suppressing inappropriate confidences for unknown ones. Experimental results demonstrate that OpenViewer effectively addresses openness challenges while ensuring recognition performance for both known and unknown samples. The code is released at https://github.com/dushide/OpenViewer.