ASPIRE: Assistive System for Performance Evaluation in IR

TL;DR

ASPIRE is a visual analytics tool that enhances the evaluation and comparison of IR models across multiple dimensions, aiding researchers in understanding complex IR experiment results.

Contribution

It introduces a comprehensive, user-friendly system for in-depth IR performance analysis, supporting multi-faceted comparisons and query-level insights.

Findings

Supports detailed IR experiment comparisons

Enables query-level and collection-based analysis

Showcases effectiveness with TREC Clinical Trials data

Abstract

Information Retrieval (IR) evaluation involves far more complexity than merely presenting performance measures in a table. Researchers often need to compare multiple models across various dimensions, such as the Precision-Recall trade-off and response time, to understand the reasons behind the varying performance of specific queries for different models. We introduce ASPIRE (Assistive System for Performance Evaluation in IR), a visual analytics tool designed to address these complexities by providing an extensive and user-friendly interface for in-depth analysis of IR experiments. ASPIRE supports four key aspects of IR experiment evaluation and analysis: single/multi-experiment comparisons, query-level analysis, query characteristics-performance interplay, and collection-based retrieval analysis. We showcase the functionality of ASPIRE using the TREC Clinical Trials collection. ASPIRE is an open-source toolkit available online: https://github.com/GiorgosPeikos/ASPIRE