BlazePose: On-device Real-time Body Pose tracking

TL;DR

BlazePose is a lightweight neural network designed for real-time human body pose estimation on mobile devices, achieving over 30 fps and suitable for applications like fitness and sign language recognition.

Contribution

It introduces a novel, efficient neural network architecture that combines heatmaps and regression for accurate, real-time body pose tracking on mobile hardware.

Findings

Runs at over 30 fps on a Pixel 2

Estimates 33 body keypoints for a single person

Suitable for real-time mobile applications

Abstract

We present BlazePose, a lightweight convolutional neural network architecture for human pose estimation that is tailored for real-time inference on mobile devices. During inference, the network produces 33 body keypoints for a single person and runs at over 30 frames per second on a Pixel 2 phone. This makes it particularly suited to real-time use cases like fitness tracking and sign language recognition. Our main contributions include a novel body pose tracking solution and a lightweight body pose estimation neural network that uses both heatmaps and regression to keypoint coordinates.