CROSS-JEM: Accurate and Efficient Cross-encoders for Short-text Ranking Tasks

TL;DR

CROSS-JEM is a novel transformer-based ranking model that jointly scores multiple items per query, significantly improving accuracy and reducing latency in search and recommendation tasks.

Contribution

We introduce CROSS-JEM, a joint scoring approach for transformers that enhances efficiency and accuracy in ranking multiple short-text items simultaneously.

Findings

Achieves state-of-the-art accuracy on public datasets.

Over 4x reduction in ranking latency.

Effectively models multiple items per query.

Abstract

Ranking a set of items based on their relevance to a given query is a core problem in search and recommendation. Transformer-based ranking models are the state-of-the-art approaches for such tasks, but they score each query-item independently, ignoring the joint context of other relevant items. This leads to sub-optimal ranking accuracy and high computational costs. In response, we propose Cross-encoders with Joint Efficient Modeling (CROSS-JEM), a novel ranking approach that enables transformer-based models to jointly score multiple items for a query, maximizing parameter utilization. CROSS-JEM leverages (a) redundancies and token overlaps to jointly score multiple items, that are typically short-text phrases arising in search and recommendations, and (b) a novel training objective that models ranking probabilities. CROSS-JEM achieves state-of-the-art accuracy and over 4x lower ranking latency over standard cross-encoders. Our contributions are threefold: (i) we highlight the gap between the ranking application's need for scoring thousands of items per query and the limited capabilities of current cross-encoders; (ii) we introduce CROSS-JEM for joint efficient scoring of multiple items per query; and (iii) we demonstrate state-of-the-art accuracy on standard public datasets and a proprietary dataset. CROSS-JEM opens up new directions for designing tailored early-attention-based ranking models that incorporate strict production constraints such as item multiplicity and latency.