ReSIM: Re-ranking Binary Similarity Embeddings to Improve Function Search Performance

TL;DR

ReSIM enhances binary function similarity search by adding a neural re-ranker that jointly assesses query-candidate pairs, significantly improving retrieval accuracy over traditional embedding-only methods.

Contribution

This paper introduces ReSIM, a neural re-ranking system that improves binary function search by capturing cross-function relationships, a novel approach in this domain.

Findings

ReSIM achieves an average of 21.7% improvement in nDCG.

ReSIM achieves an average of 27.8% improvement in Recall.

Re-ranking consistently enhances search effectiveness across models and datasets.

Abstract

Binary Function Similarity (BFS), the problem of determining whether two binary functions originate from the same source code, has been extensively studied in recent research across security, software engineering, and machine learning communities. This interest arises from its central role in developing vulnerability detection systems, copyright infringement analysis, and malware phylogeny tools. Nearly all binary function similarity systems embed assembly functions into real-valued vectors, where similar functions map to points that lie close to each other in the metric space. These embeddings enable function search: a query function is embedded and compared against a database of candidate embeddings to retrieve the most similar matches. Despite their effectiveness, such systems rely on bi-encoder architectures that embed functions independently, limiting their ability to capture cross-function relationships and similarities. To address this limitation, we introduce ReSIM, a novel and enhanced function search system that complements embedding-based search with a neural re-ranker. Unlike traditional embedding models, our reranking module jointly processes query-candidate pairs to compute ranking scores based on their mutual representation, allowing for more accurate similarity assessment. By re-ranking the top results from embedding-based retrieval, ReSIM leverages fine-grained relation information that bi-encoders cannot capture. We evaluate ReSIM across seven embedding models on two benchmark datasets, demonstrating consistent improvements in search effectiveness, with average gains of 21.7% in terms of nDCG and 27.8% in terms of Recall.