# Zero-shot Learning of 3D Point Cloud Objects

## TL;DR

This paper introduces the first zero-shot learning framework for 3D point cloud object classification, enabling recognition of unseen classes by leveraging semantic information, and establishes a new evaluation protocol for this task.

## Contribution

It adapts existing 3D recognition models to the zero-shot learning setting and proposes a standard evaluation protocol for unseen class recognition in 3D point clouds.

## Key findings

- Baseline performances established on new protocol
- First application of ZSL to 3D point cloud objects
- Highlights challenges and future directions for ZSL in 3D data

## Abstract

Recent deep learning architectures can recognize instances of 3D point cloud objects of previously seen classes quite well. At the same time, current 3D depth camera technology allows generating/segmenting a large amount of 3D point cloud objects from an arbitrary scene, for which there is no previously seen training data. A challenge for a 3D point cloud recognition system is, then, to classify objects from new, unseen, classes. This issue can be resolved by adopting a zero-shot learning (ZSL) approach for 3D data, similar to the 2D image version of the same problem. ZSL attempts to classify unseen objects by comparing semantic information (attribute/word vector) of seen and unseen classes. Here, we adapt several recent 3D point cloud recognition systems to the ZSL setting with some changes to their architectures. To the best of our knowledge, this is the first attempt to classify unseen 3D point cloud objects in the ZSL setting. A standard protocol (which includes the choice of datasets and the seen/unseen split) to evaluate such systems is also proposed. Baseline performances are reported using the new protocol on the investigated models. This investigation throws a new challenge to the 3D point cloud recognition community that may instigate numerous future works.