SkipClick: Combining Quick Responses and Low-Level Features for Interactive Segmentation in Winter Sports Contexts

TL;DR

This paper introduces SkipClick, an interactive segmentation architecture optimized for winter sports objects, achieving state-of-the-art results by combining quick response mechanisms with low-level features.

Contribution

The paper proposes a novel architecture for interactive segmentation tailored to winter sports, improving response speed and segmentation accuracy over existing models.

Findings

Outperforms SAM and HQ-SAM on WSESeg dataset.

Achieves state-of-the-art results on HQSeg-44k dataset.

Effective on a new skiing human masks dataset.

Abstract

In this paper, we present a novel architecture for interactive segmentation in winter sports contexts. The field of interactive segmentation deals with the prediction of high-quality segmentation masks by informing the network about the objects position with the help of user guidance. In our case the guidance consists of click prompts. For this task, we first present a baseline architecture which is specifically geared towards quickly responding after each click. Afterwards, we motivate and describe a number of architectural modifications which improve the performance when tasked with segmenting winter sports equipment on the WSESeg dataset. With regards to the average NoC@85 metric on the WSESeg classes, we outperform SAM and HQ-SAM by 2.336 and 7.946 clicks, respectively. When applied to the HQSeg-44k dataset, our system delivers state-of-the-art results with a NoC@90 of 6.00 and NoC@95 of 9.89. In addition to that, we test our model on a novel dataset containing masks for humans during skiing.