# Learnable Triangulation of Human Pose

## TL;DR

This paper introduces two innovative learnable triangulation methods for multi-view 3D human pose estimation, enhancing accuracy and transferability across datasets by enabling end-to-end training and incorporating volumetric aggregation.

## Contribution

The paper proposes two novel differentiable triangulation approaches, including a volumetric aggregation method, for improved multi-view 3D human pose estimation.

## Key findings

- Significant improvement over state-of-the-art on Human3.6M dataset
- Methods are end-to-end differentiable and transferable across datasets
- Volumetric aggregation enhances 3D pose accuracy

## Abstract

We present two novel solutions for multi-view 3D human pose estimation based on new learnable triangulation methods that combine 3D information from multiple 2D views. The first (baseline) solution is a basic differentiable algebraic triangulation with an addition of confidence weights estimated from the input images. The second solution is based on a novel method of volumetric aggregation from intermediate 2D backbone feature maps. The aggregated volume is then refined via 3D convolutions that produce final 3D joint heatmaps and allow modelling a human pose prior. Crucially, both approaches are end-to-end differentiable, which allows us to directly optimize the target metric. We demonstrate transferability of the solutions across datasets and considerably improve the multi-view state of the art on the Human3.6M dataset. Video demonstration, annotations and additional materials will be posted on our project page (https://saic-violet.github.io/learnable-triangulation).