CARLoS: Retrieval via Concise Assessment Representation of LoRAs at Scale

TL;DR

CARLoS introduces a scalable, metadata-free framework for characterizing and retrieving LoRAs based on their semantic effects, stability, and strength, improving usability and legal analysis.

Contribution

The paper presents a novel concise representation for LoRAs using CLIP embeddings, enabling effective large-scale retrieval without extra metadata.

Findings

Outperforms textual baselines in retrieval accuracy.

Effectively filters unstable or overly strong LoRAs.

Supports legal analysis of LoRAs' substantiality and volition.

Abstract

The rapid proliferation of generative components, such as LoRAs, has created a vast but unstructured ecosystem. Existing discovery methods depend on unreliable user descriptions or biased popularity metrics, hindering usability. We present CARLoS, a large-scale framework for characterizing LoRAs without requiring additional metadata. Analyzing over 650 LoRAs, we employ them in image generation over a variety of prompts and seeds, as a credible way to assess their behavior. Using CLIP embeddings and their difference to a base-model generation, we concisely define a three-part representation: Directions, defining semantic shift; Strength, quantifying the significance of the effect; and Consistency, quantifying how stable the effect is. Using these representations, we develop an efficient retrieval framework that semantically matches textual queries to relevant LoRAs while filtering overly strong or unstable ones, outperforming textual baselines in automated and human evaluations. While retrieval is our primary focus, the same representation also supports analyses linking Strength and Consistency to legal notions of substantiality and volition, key considerations in copyright, positioning CARLoS as a practical system with broader relevance for LoRA analysis.