SketchQL Demonstration: Zero-shot Video Moment Querying with Sketches

TL;DR

SketchQL is a novel video database system enabling zero-shot retrieval of video moments through a sketch-based interface that allows users to specify object trajectories and compose complex events.

Contribution

It introduces a new sketch-based query interface combined with a pre-trained trajectory similarity model for zero-shot video moment retrieval.

Findings

Effective zero-shot retrieval of video moments.

User-friendly sketch-based query interface.

Successful demonstration with real-world scenarios.

Abstract

In this paper, we will present SketchQL, a video database management system (VDBMS) for retrieving video moments with a sketch-based query interface. This novel interface allows users to specify object trajectory events with simple mouse drag-and-drop operations. Users can use trajectories of single objects as building blocks to compose complex events. Using a pre-trained model that encodes trajectory similarity, SketchQL achieves zero-shot video moments retrieval by performing similarity searches over the video to identify clips that are the most similar to the visual query. In this demonstration, we introduce the graphic user interface of SketchQL and detail its functionalities and interaction mechanisms. We also demonstrate the end-to-end usage of SketchQL from query composition to video moments retrieval using real-world scenarios.