FIFA: Fast Inference Approximation for Action Segmentation

TL;DR

FIFA introduces a fast, approximate, differentiable inference method for action segmentation that significantly speeds up processing while maintaining high accuracy, outperforming traditional exact inference methods.

Contribution

FIFA provides a novel, general approximate inference approach that replaces dynamic programming, achieving over five times faster inference with comparable performance.

Findings

FIFA speeds up inference by more than 5 times.

Maintains state-of-the-art accuracy on action segmentation datasets.

Applicable to both weakly and fully supervised settings.

Abstract

We introduce FIFA, a fast approximate inference method for action segmentation and alignment. Unlike previous approaches, FIFA does not rely on expensive dynamic programming for inference. Instead, it uses an approximate differentiable energy function that can be minimized using gradient-descent. FIFA is a general approach that can replace exact inference improving its speed by more than 5 times while maintaining its performance. FIFA is an anytime inference algorithm that provides a better speed vs. accuracy trade-off compared to exact inference. We apply FIFA on top of state-of-the-art approaches for weakly supervised action segmentation and alignment as well as fully supervised action segmentation. FIFA achieves state-of-the-art results on most metrics on two action segmentation datasets.