LTSim: Layout Transportation-based Similarity Measure for Evaluating Layout Generation

TL;DR

This paper introduces LTSim, a novel layout similarity measure based on optimal transport, enabling flexible comparison of layouts regardless of element categories, and demonstrating superior reliability over existing measures like FID.

Contribution

The paper proposes a new layout similarity measure using optimal transport that overcomes limitations of previous methods dependent on strict element matching.

Findings

LTSim effectively compares layouts with no shared element categories.

Collection-level LTSim provides more reliable group layout comparisons.

Empirical results show LTSim outperforms FID and Max.IoU in reliability.

Abstract

We introduce a layout similarity measure designed to evaluate the results of layout generation. While several similarity measures have been proposed in prior research, there has been a lack of comprehensive discussion about their behaviors. Our research uncovers that the majority of these measures are unable to handle various layout differences, primarily due to their dependencies on strict element matching, that is one-by-one matching of elements within the same category. To overcome this limitation, we propose a new similarity measure based on optimal transport, which facilitates a more flexible matching of elements. This approach allows us to quantify the similarity between any two layouts even those sharing no element categories, making our measure highly applicable to a wide range of layout generation tasks. For tasks such as unconditional layout generation, where FID is commonly used, we also extend our measure to deal with collection-level similarities between groups of layouts. The empirical result suggests that our collection-level measure offers more reliable comparisons than existing ones like FID and Max.IoU.