Picasso: A CUDA-based Library for Deep Learning over 3D Meshes

TL;DR

Picasso is a CUDA-based library that enables efficient deep learning on 3D meshes by introducing novel GPU-accelerated modules for mesh decimation, convolution, and robustness to sampling variations, improving multi-scale feature extraction.

Contribution

The paper introduces new GPU-accelerated mesh decimation and convolution modules, along with a fuzzy mechanism, for deep learning over 3D meshes, addressing limitations of CPU-based methods.

Findings

Achieved competitive segmentation results on S3DIS dataset.

Demonstrated efficient on-the-fly mesh decimation on GPU.

Provided a publicly available library for deep learning on 3D meshes.

Abstract

We present Picasso, a CUDA-based library comprising novel modules for deep learning over complex real-world 3D meshes. Hierarchical neural architectures have proved effective in multi-scale feature extraction which signifies the need for fast mesh decimation. However, existing methods rely on CPU-based implementations to obtain multi-resolution meshes. We design GPU-accelerated mesh decimation to facilitate network resolution reduction efficiently on-the-fly. Pooling and unpooling modules are defined on the vertex clusters gathered during decimation. For feature learning over meshes, Picasso contains three types of novel convolutions namely, facet2vertex, vertex2facet, and facet2facet convolution. Hence, it treats a mesh as a geometric structure comprising vertices and facets, rather than a spatial graph with edges as previous methods do. Picasso also incorporates a fuzzy mechanism in its filters for robustness to mesh sampling (vertex density). It exploits Gaussian mixtures to define fuzzy coefficients for the facet2vertex convolution, and barycentric interpolation to define the coefficients for the remaining two convolutions. In this release, we demonstrate the effectiveness of the proposed modules with competitive segmentation results on S3DIS. The library will be made public through https://github.com/hlei-ziyan/Picasso.