Context-Aware Streaming Perception in Dynamic Environments
Gur-Eyal Sela, Ionel Gog, Justin Wong, Kumar Krishna Agrawal, Xiangxi, Mo, Sukrit Kalra, Peter Schafhalter, Eric Leong, Xin Wang, Bharathan Balaji,, Joseph Gonzalez, Ion Stoica

TL;DR
This paper introduces Octopus, a method that dynamically adapts vision system configurations to maximize streaming accuracy in real-time environments, accounting for scene difficulty and obstacle movement.
Contribution
It proposes a novel approach to optimize streaming accuracy for each environment context, improving real-time perception performance in dynamic scenes.
Findings
Improves tracking performance (S-MOTA) by 7.4% over static methods.
Maximizes accuracy in streaming settings by considering scene difficulty and obstacle displacement.
Enhances real-time vision systems without compromising offline accuracy.
Abstract
Efficient vision works maximize accuracy under a latency budget. These works evaluate accuracy offline, one image at a time. However, real-time vision applications like autonomous driving operate in streaming settings, where ground truth changes between inference start and finish. This results in a significant accuracy drop. Therefore, a recent work proposed to maximize accuracy in streaming settings on average. In this paper, we propose to maximize streaming accuracy for every environment context. We posit that scenario difficulty influences the initial (offline) accuracy difference, while obstacle displacement in the scene affects the subsequent accuracy degradation. Our method, Octopus, uses these scenario properties to select configurations that maximize streaming accuracy at test time. Our method improves tracking performance (S-MOTA) by 7.4% over the conventional static approach.…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsVisual Attention and Saliency Detection · Advanced Vision and Imaging · Advanced Optical Sensing Technologies
MethodsTest
