Multiresolution hierarchy co-clustering for semantic segmentation in sequences with small variations

TL;DR

This paper introduces a multiresolution co-clustering method for semantic segmentation in image sequences with small variations, utilizing hierarchies and quadratic semi-assignment optimization to achieve state-of-the-art results.

Contribution

It formalizes a co-clustering approach as a quadratic semi-assignment problem and extends it to a multiresolution video segmentation framework.

Findings

Achieves state-of-the-art segmentation results on the Video Occlusion/Object Boundary Detection Dataset.

Effectively clusters hierarchies for coherent multiresolution representations.

Demonstrates the effectiveness of the approach in sequences with small variations.

Abstract

This paper presents a co-clustering technique that, given a collection of images and their hierarchies, clusters nodes from these hierarchies to obtain a coherent multiresolution representation of the image collection. We formalize the co-clustering as a Quadratic Semi-Assignment Problem and solve it with a linear programming relaxation approach that makes effective use of information from hierarchies. Initially, we address the problem of generating an optimal, coherent partition per image and, afterwards, we extend this method to a multiresolution framework. Finally, we particularize this framework to an iterative multiresolution video segmentation algorithm in sequences with small variations. We evaluate the algorithm on the Video Occlusion/Object Boundary Detection Dataset, showing that it produces state-of-the-art results in these scenarios.