Empowering Knowledge Distillation via Open Set Recognition for Robust 3D Point Cloud Classification

TL;DR

This paper introduces a novel training method combining knowledge distillation and open set recognition to improve the robustness of small 3D point cloud classifiers in real-world scenarios.

Contribution

It proposes a joint training approach for 3D object recognition that integrates knowledge distillation with open set recognition, addressing a gap in existing methods.

Findings

Smaller models with minimal performance loss

Effective open set recognition for 3D point clouds

Enhanced robustness in real-world scenarios

Abstract

Real-world scenarios pose several challenges to deep learning based computer vision techniques despite their tremendous success in research. Deeper models provide better performance, but are challenging to deploy and knowledge distillation allows us to train smaller models with minimal loss in performance. The model also has to deal with open set samples from classes outside the ones it was trained on and should be able to identify them as unknown samples while classifying the known ones correctly. Finally, most existing image recognition research focuses only on using two-dimensional snapshots of the real world three-dimensional objects. In this work, we aim to bridge these three research fields, which have been developed independently until now, despite being deeply interrelated. We propose a joint Knowledge Distillation and Open Set recognition training methodology for three-dimensional object recognition. We demonstrate the effectiveness of the proposed method via various experiments on how it allows us to obtain a much smaller model, which takes a minimal hit in performance while being capable of open set recognition for 3D point cloud data.