NeuralNetwork-Viterbi: A Framework for Weakly Supervised Video Learning

TL;DR

This paper introduces NeuralNetwork-Viterbi, a framework that enables online weakly supervised video learning using a Viterbi-based loss, significantly improving segmentation accuracy by modeling context and length.

Contribution

The work presents a novel Viterbi-based loss for weakly supervised video learning, incorporating explicit context and length modeling for better segmentation.

Findings

Up to 10% improvement over state-of-the-art methods on action segmentation benchmarks.

Effective online and incremental learning from weakly annotated video data.

Enhanced video segmentation accuracy through context and length modeling.

Abstract

Video learning is an important task in computer vision and has experienced increasing interest over the recent years. Since even a small amount of videos easily comprises several million frames, methods that do not rely on a frame-level annotation are of special importance. In this work, we propose a novel learning algorithm with a Viterbi-based loss that allows for online and incremental learning of weakly annotated video data. We moreover show that explicit context and length modeling leads to huge improvements in video segmentation and labeling tasks andinclude these models into our framework. On several action segmentation benchmarks, we obtain an improvement of up to 10% compared to current state-of-the-art methods.