TL;DR
VINS is a visual search framework that enables retrieval of similar mobile UI designs based on structural and visual features, aiding designers in inspiration and comparison.
Contribution
Introduces VINS, a novel UI image retrieval system that incorporates UI structure and content for more accurate search results.
Findings
Achieves 76.39% mean Average Precision in UI detection
Develops a large-scale UI dataset with detailed view hierarchy
Demonstrates high performance in retrieving similar UI designs
Abstract
Searching for relative mobile user interface (UI) design examples can aid interface designers in gaining inspiration and comparing design alternatives. However, finding such design examples is challenging, especially as current search systems rely on only text-based queries and do not consider the UI structure and content into account. This paper introduces VINS, a visual search framework, that takes as input a UI image (wireframe, high-fidelity) and retrieves visually similar design examples. We first survey interface designers to better understand their example finding process. We then develop a large-scale UI dataset that provides an accurate specification of the interface's view hierarchy (i.e., all the UI components and their specific location). By utilizing this dataset, we propose an object-detection based image retrieval framework that models the UI context and hierarchical…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
