4D Association Graph for Realtime Multi-person Motion Capture Using Multiple Video Cameras
Yuxiang Zhang, Liang An, Tao Yu, Xiu Li, Kun Li, Yebin Liu

TL;DR
This paper introduces a real-time multi-person motion capture system using a novel 4D association graph that unifies parsing, matching, and tracking across multiple views and time, achieving high accuracy and efficiency.
Contribution
It presents the first unified 4D association graph framework for real-time multi-view, multi-person motion capture, with a new efficient optimization method and a multiview dataset.
Findings
Runs at 30fps with 5 cameras on 5 persons
Outperforms state-of-the-art methods quantitatively
Robust to noisy detections
Abstract
This paper contributes a novel realtime multi-person motion capture algorithm using multiview video inputs. Due to the heavy occlusions in each view, joint optimization on the multiview images and multiple temporal frames is indispensable, which brings up the essential challenge of realtime efficiency. To this end, for the first time, we unify per-view parsing, cross-view matching, and temporal tracking into a single optimization framework, i.e., a 4D association graph that each dimension (image space, viewpoint and time) can be treated equally and simultaneously. To solve the 4D association graph efficiently, we further contribute the idea of 4D limb bundle parsing based on heuristic searching, followed with limb bundle assembling by proposing a bundle Kruskal's algorithm. Our method enables a realtime online motion capture system running at 30fps using 5 cameras on a 5-person scene.…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
4D Association Graph for Realtime Multi-Person Motion Capture Using Multiple Video Cameras· youtube
Taxonomy
TopicsHuman Pose and Action Recognition · Advanced Vision and Imaging · Video Surveillance and Tracking Methods
