Advances and Trends in the 3D Reconstruction of the Shape and Motion of Animals

TL;DR

This survey reviews recent deep learning methods for non-intrusive 3D animal shape and motion reconstruction from images and videos, highlighting advances, challenges, and future research directions.

Contribution

It categorizes and analyzes the latest techniques in 3D animal reconstruction, providing a comprehensive overview of the field's state-of-the-art and identifying key challenges.

Findings

Deep learning enables non-intrusive 3D animal reconstruction from RGB data.

Current methods vary in input modalities, representations, and training mechanisms.

The survey highlights existing limitations and suggests future research directions.

Abstract

Reconstructing the 3D geometry, pose, and motion of animals is a long-standing problem, which has a wide range of applications, from biology, livestock management, and animal conservation and welfare to content creation in digital entertainment and Virtual/Augmented Reality (VR/AR). Traditionally, 3D models of real animals are obtained using 3D scanners. These, however, are intrusive, often prohibitively expensive, and difficult to deploy in the natural environment of the animals. In recent years, we have seen a significant surge in deep learning-based techniques that enable the 3D reconstruction, in a non-intrusive manner, of the shape and motion of dynamic objects just from their RGB image and/or video observations. Several papers have explored their application and extension to various types of animals. This paper surveys the latest developments in this emerging and growing field of research. It categorizes and discusses the state-of-the-art methods based on their input modalities, the way the 3D geometry and motion of animals are represented, the type of reconstruction techniques they use, and the training mechanisms they adopt. It also analyzes the performance of some key methods, discusses their strengths and limitations, and identifies current challenges and directions for future research.