Timestamp-independent Haptic-Visual Synchronization
Yiwen Xu, Liangtao Huang, Tiesong Zhao, Liqun Lin, Ying Fang

TL;DR
This paper introduces a timestamp-independent method for synchronizing haptic and visual signals in multimedia systems, enhancing robustness and user experience without relying on timestamps.
Contribution
It proposes a novel timestamp-independent synchronization model using key sample and frame extraction based on signal characteristics and deep object detection.
Findings
Effective synchronization without timestamps
Improved robustness under complex network conditions
Enhanced user experience in multimedia interactions
Abstract
The booming haptic data significantly improves the users'immersion during multimedia interaction. As a result, the study of Haptic,Audio-Visual Environment(HAVE)has attracted attentions of multimedia community. To realize such a system, a challenging tack is the synchronization of multiple sensorial signals that is critical to user experience. Despite of audio-visual synchronization efforts, there is still a lack of haptic-aware multimedia synchronization model. In this work, we propose a timestamp-independent synchronization for haptic-visual signal transmission. First, we exploit the sequential correlations during delivery and playback of a haptic-visual communication system. Second, we develop a key sample extraction of haptic signals based on the force feedback characteristics, and a key frame extraction of visual signals based on deep object detection. Third, we combine the key…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsTeleoperation and Haptic Systems · Multimedia Communication and Technology · Neural Networks and Reservoir Computing
