A Feature Clustering Approach Based on Histogram of Oriented Optical Flow and Superpixels

TL;DR

This paper introduces a novel feature clustering method using superpixels and short-term Histogram of Oriented Optical Flow, effectively segmenting static and moving objects in videos with heavy camera movements without prior object count knowledge.

Contribution

The paper proposes a new clustering approach combining superpixels, SDPF features, and HOOF for improved video object segmentation, overcoming limitations of existing algorithms.

Findings

Outperforms K-Means-based clustering in accuracy metrics

Successfully segments unknown number of static and moving objects

Enhances spatial consistency and semantic segmentation quality

Abstract

Visual feature clustering is one of the cost-effective approaches to segment objects in videos. However, the assumptions made for developing the existing algorithms prevent them from being used in situations like segmenting an unknown number of static and moving objects under heavy camera movements. This paper addresses the problem by introducing a clustering approach based on superpixels and short-term Histogram of Oriented Optical Flow (HOOF). Salient Dither Pattern Feature (SDPF) is used as the visual feature to track the flow and Simple Linear Iterative Clustering (SLIC) is used for obtaining the superpixels. This new clustering approach is based on merging superpixels by comparing short term local HOOF and a color cue to form high-level semantic segments. The new approach was compared with one of the latest feature clustering approaches based on K-Means in eight-dimensional space and the results revealed that the new approach is better by means of consistency, completeness, and spatial accuracy. Further, the new approach completely solved the problem of not knowing the number of objects in a scene.