# Practical User Feedback-driven Internal Search Using Online Learning to Rank

## TL;DR

Spoke is a SaaS-based internal search system that uses conversational user feedback and online learning-to-rank algorithms to continually improve relevance tailored to each organization, outperforming standard systems.

## Contribution

Introducing Spoke, a practical, feedback-driven internal search system that leverages real-time online learning-to-rank for personalized relevance without explicit domain knowledge.

## Key findings

- Spoke outperforms baseline systems by up to 41% in offline F1 scores.
- The system effectively adapts relevance scoring to diverse organizational domains.
- Practical considerations enable easy deployment and lifecycle management.

## Abstract

We present a system, Spoke, for creating and searching internal knowledge base (KB) articles for organizations. Spoke is available as a SaaS (Software-as-a-Service) product deployed across hundreds of organizations with a diverse set of domains. Spoke continually improves search quality using conversational user feedback which allows it to provide better search experience than standard information retrieval systems without encoding any explicit domain knowledge. We achieve this by using a real-time online learning-to-rank (L2R) algorithm that automatically customizes relevance scoring for each organization deploying Spoke by using a query similarity kernel. The focus of this paper is on incorporating practical considerations into our relevance scoring function and algorithm that make Spoke easy to deploy and suitable for handling events that naturally happen over the life-cycle of any KB deployment. We show that Spoke outperforms competitive baselines by up to 41% in offline F1 comparisons.