DSFlash: Comprehensive Panoptic Scene Graph Generation in Realtime
Julian Lorenz, Vladyslav Kovganko, Elias Kohout, Mrunmai Phatak, Daniel Kienzle, Rainer Lienhart

TL;DR
DSFlash is a fast, resource-efficient panoptic scene graph generation model capable of processing video streams at 56 fps, providing comprehensive scene understanding suitable for real-time applications on limited hardware.
Contribution
The paper introduces DSFlash, a low-latency, resource-efficient model that generates comprehensive scene graphs in real-time, addressing deployment challenges on edge devices.
Findings
Processes video at 56 fps on RTX 3090
Requires less than 24 hours to train on GTX 1080
Generates richer, comprehensive scene graphs
Abstract
Scene Graph Generation (SGG) aims to extract a detailed graph structure from an image, a representation that holds significant promise as a robust intermediate step for complex downstream tasks like reasoning for embodied agents. However, practical deployment in real-world applications - especially on resource constrained edge devices - requires speed and resource efficiency, challenges that have received limited attention in existing research. To bridge this gap, we introduce DSFlash, a low-latency model for panoptic scene graph generation designed to overcome these limitations. DSFlash can process a video stream at 56 frames per second on a standard RTX 3090 GPU, without compromising performance against existing state-of-the-art methods. Crucially, unlike prior approaches that often restrict themselves to salient relationships, DSFlash computes comprehensive scene graphs, offering…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsMultimodal Machine Learning Applications · Generative Adversarial Networks and Image Synthesis · Graph Theory and Algorithms
