Kaolin: A PyTorch Library for Accelerating 3D Deep Learning Research

TL;DR

Kaolin is an open-source PyTorch library that accelerates 3D deep learning research by providing differentiable modules, dataset handling, visualization, and a model zoo for various 3D representations.

Contribution

It introduces a comprehensive, easy-to-use library that integrates 3D data processing, differentiable graphics, and a collection of state-of-the-art models for 3D deep learning research.

Findings

Streamlines 3D deep learning workflows

Enables efficient model development and evaluation

Supports multiple 3D data formats and tasks

Abstract

We present Kaolin, a PyTorch library aiming to accelerate 3D deep learning research. Kaolin provides efficient implementations of differentiable 3D modules for use in deep learning systems. With functionality to load and preprocess several popular 3D datasets, and native functions to manipulate meshes, pointclouds, signed distance functions, and voxel grids, Kaolin mitigates the need to write wasteful boilerplate code. Kaolin packages together several differentiable graphics modules including rendering, lighting, shading, and view warping. Kaolin also supports an array of loss functions and evaluation metrics for seamless evaluation and provides visualization functionality to render the 3D results. Importantly, we curate a comprehensive model zoo comprising many state-of-the-art 3D deep learning architectures, to serve as a starting point for future research endeavours. Kaolin is available as open-source software at https://github.com/NVIDIAGameWorks/kaolin/.