Using modern motion estimation algorithms in existing video codecs

TL;DR

This paper compares various modern motion estimation algorithms within H.264 and MP4 codecs, highlighting the complex relationship between motion estimation methods and compression efficiency, and exploring adaptation techniques for pre-computed motion fields.

Contribution

It provides a comparative analysis of recent motion estimation algorithms in video codecs and introduces methods for adapting pre-computed motion fields for improved compression.

Findings

No significant rate-distortion gains observed with chosen methods

Complex interrelationship between motion estimation and compression

Initial results suggest further exploration is valuable

Abstract

Motion estimation is a key component of any modern video codec. Our understanding of motion and the estimation of motion from video has come a very long way since 2000. More than 135 different algorithms have been recently reviewed by Scharstein et al http://vision.middlebury.edu/flow/. These new algorithms differ markedly from Block Matching which has been the mainstay of video compression for some time. This paper presents comparisons of H.264 and MP4 compression using different motion estimation methods. In so doing we present as well methods for adapting pre-computed motion fields for use within a codec. We do not observe significant gains to be had with the methods chosen w.r.t. Rate Distortion tradeoffs but the results reflect a significantly more complex interrelationship between motion and compression than would be expected. There remains much more to be done to improve the coverage of this comparison to the emerging standards but these initial results show that there is value in these explorations.