Sheaf-Laplacian Obstruction and Projection Hardness for Cross-Modal Compatibility on a Modality-Independent Site

TL;DR

This paper introduces a unified sheaf-theoretic framework to analyze cross-modal compatibility, identifying two key failure modes—projection hardness and sheaf-Laplacian obstruction—and providing tools for their quantification and mitigation.

Contribution

It formalizes a novel sheaf-based approach to understand and quantify the intrinsic incompatibilities in learned cross-modal representations, including bounds and constructions for compatibility analysis.

Findings

Sheaf-Laplacian obstruction energy correlates with alignment error.

Compatibility failure can be due to hardness or obstruction, which are separable.

Intermediate modalities can reduce hardness even if direct alignment fails.

Abstract

We develop a unified framework for analyzing cross-modal compatibility in learned representations. The core object is a modality-independent neighborhood site on sample indices, equipped with a cellular sheaf of finite-dimensional real inner-product spaces. For a directed modality pair $(a\to b)$, we formalize two complementary incompatibility mechanisms: projection hardness, the minimal complexity within a nested Lipschitz-controlled projection family needed for a single global map to align whitened embeddings; and sheaf-Laplacian obstruction, the minimal spatial variation required by a locally fit field of projection parameters to achieve a target alignment error. The obstruction invariant is implemented via a projection-parameter sheaf whose 0-Laplacian energy exactly matches the smoothness penalty used in sheaf-regularized regression, making the theory directly operational. This separates two distinct failure modes: hardness failure, where no low-complexity global projection exists, and obstruction failure, where local projections exist but cannot be made globally consistent over the semantic neighborhood graph without large parameter variation. We link the sheaf spectral gap to stability of global alignment, derive bounds relating obstruction energy to excess global-map error under mild Lipschitz assumptions, and give explicit constructions showing that compatibility is generally non-transitive. We further define bridging via composed projection families and show, in a concrete ReLU setting, that an intermediate modality can strictly reduce effective hardness even when direct alignment remains infeasible.