CLAS: A Machine Learning Enhanced Framework for Exploring Large 3D Design Datasets

TL;DR

This paper introduces CLAS, a machine learning framework that enables automatic retrieval and synthesis of 3D objects from large datasets, aiding designers in efficiently accessing and utilizing 3D models for creative and training purposes.

Contribution

The paper presents a novel ML-enhanced framework, CLAS, for automatic retrieval of 3D objects based on user specifications, improving access to large 3D datasets.

Findings

Achieved a mean reciprocal rank of 0.58 in retrieval accuracy.

Top 1 accuracy of 42.27% in the retrieval system.

System successfully retrieved 6,778 chair models from ShapeNet.

Abstract

Three-dimensional (3D) objects have wide applications. Despite the growing interest in 3D modeling in academia and industries, designing and/or creating 3D objects from scratch remains time-consuming and challenging. With the development of generative artificial intelligence (AI), designers discover a new way to create images for ideation. However, generative AIs are less useful in creating 3D objects with satisfying qualities. To allow 3D designers to access a wide range of 3D objects for creative activities based on their specific demands, we propose a machine learning (ML) enhanced framework CLAS - named after the four-step of capture, label, associate, and search - to enable fully automatic retrieval of 3D objects based on user specifications leveraging the existing datasets of 3D objects. CLAS provides an effective and efficient method for any person or organization to benefit from their existing but not utilized 3D datasets. In addition, CLAS may also be used to produce high-quality 3D object synthesis datasets for training and evaluating 3D generative models. As a proof of concept, we created and showcased a search system with a web user interface (UI) for retrieving 6,778 3D objects of chairs in the ShapeNet dataset powered by CLAS. In a close-set retrieval setting, our retrieval method achieves a mean reciprocal rank (MRR) of 0.58, top 1 accuracy of 42.27%, and top 10 accuracy of 89.64%.