AIpparel: A Multimodal Foundation Model for Digital Garments

TL;DR

AIpparel is a multimodal foundation model that simplifies garment creation by generating and editing sewing patterns using a large-scale dataset and novel tokenization, enabling advanced text and image-based garment tasks.

Contribution

The paper introduces a new multimodal model with a specialized tokenization scheme for sewing patterns, achieving state-of-the-art results in garment generation and editing.

Findings

State-of-the-art performance in text-to-garment prediction

Effective multimodal garment editing capabilities

Successful encoding of complex sewing patterns

Abstract

Apparel is essential to human life, offering protection, mirroring cultural identities, and showcasing personal style. Yet, the creation of garments remains a time-consuming process, largely due to the manual work involved in designing them. To simplify this process, we introduce AIpparel, a multimodal foundation model for generating and editing sewing patterns. Our model fine-tunes state-of-the-art large multimodal models (LMMs) on a custom-curated large-scale dataset of over 120,000 unique garments, each with multimodal annotations including text, images, and sewing patterns. Additionally, we propose a novel tokenization scheme that concisely encodes these complex sewing patterns so that LLMs can learn to predict them efficiently. AIpparel achieves state-of-the-art performance in single-modal tasks, including text-to-garment and image-to-garment prediction, and enables novel multimodal garment generation applications such as interactive garment editing. The project website is at https://georgenakayama.github.io/AIpparel/.