Searching Scenes by Abstracting Things
Svetlana Kordumova, Jan C. van Gemert, Cees G. M. Snoek, Arnold W. M., Smeulders

TL;DR
This paper introduces a scene representation method based on observable 'things' properties, enabling scene retrieval through linguistic descriptions and block illustrations without prior learning, demonstrating effective results.
Contribution
It presents a novel scene abstraction approach using simple object properties, translating them into linguistic and visual queries for scene discrimination without learning.
Findings
Scene retrieval is effective using only 'things' properties.
Linguistic and visual queries can discriminate scene types.
No prior learning is needed for scene retrieval.
Abstract
In this paper we propose to represent a scene as an abstraction of 'things'. We start from 'things' as generated by modern object proposals, and we investigate their immediately observable properties: position, size, aspect ratio and color, and those only. Where the recent successes and excitement of the field lie in object identification, we represent the scene composition independent of object identities. We make three contributions in this work. First, we study simple observable properties of 'things', and call it things syntax. Second, we propose translating the things syntax in linguistic abstract statements and study their descriptive effect to retrieve scenes. Thirdly, we propose querying of scenes with abstract block illustrations and study their effectiveness to discriminate among different types of scenes. The benefit of abstract statements and block illustrations is that we…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Image and Video Retrieval Techniques · Image Retrieval and Classification Techniques · Multimodal Machine Learning Applications
