SieveNet: Selecting Point-Based Features for Mesh Networks

TL;DR

SieveNet introduces a new approach for mesh neural networks that combines remeshing topology with precise geometric sampling, improving performance on 3D classification and segmentation tasks.

Contribution

It proposes a novel paradigm that integrates structured remeshing topology with accurate geometric features, eliminating the need for hand-crafted features.

Findings

Outperforms existing methods on classification tasks

Enhances segmentation accuracy

Leverages off-the-shelf architectures like vision transformers

Abstract

Meshes are widely used in 3D computer vision and graphics, but their irregular topology poses challenges in applying them to existing neural network architectures. Recent advances in mesh neural networks turn to remeshing and push the boundary of pioneer methods that solely take the raw meshes as input. Although the remeshing offers a regular topology that significantly facilitates the design of mesh network architectures, features extracted from such remeshed proxies may struggle to retain the underlying geometry faithfully, limiting the subsequent neural network's capacity. To address this issue, we propose SieveNet, a novel paradigm that takes into account both the regular topology and the exact geometry. Specifically, this method utilizes structured mesh topology from remeshing and accurate geometric information from distortion-aware point sampling on the surface of the original mesh. Furthermore, our method eliminates the need for hand-crafted feature engineering and can leverage off-the-shelf network architectures such as the vision transformer. Comprehensive experimental results on classification and segmentation tasks well demonstrate the effectiveness and superiority of our method.