Towards Semantic Search for Community Question Answering for Mortgage Officers

TL;DR

This paper presents a semantic search engine for Community Question Answering tailored to mortgage officers, utilizing NLP techniques to improve information retrieval in a complex industry.

Contribution

It introduces a hybrid, domain-adapted semantic search system combining Sentence-BERT, TF-IDF, and BM25 for improved CQA in the mortgage sector.

Findings

Hybrid search engine outperforms traditional methods

Effective domain adaptation with mortgage corpora

Improved retrieval metrics on internal dataset

Abstract

Community Question Answering (CQA) has gained increasing popularity in many domains. Mortgage is a complex and dynamic industry, and a flexible and efficient CQA platform can potentially enhance the quality of service for mortgage officers significantly. We have built a dynamic CQA platform with a state of the art semantic search engine based on recent Natural Language Processing (NLP) techniques to dynamically and collectively capture and transfer the maturity and tribal knowledge of the more experienced workforce to less experienced ones. The search engine allows for both keyword and natural language queries and is based on a fine-tuned domain-adapted Sentence-BERT encoder linearly composed with a TF-IDF vectorizer, and reciprocal-rank fused with a BM25 vectorizer. Domain adaptation and fine-tuning is based on publicly available mortgage corpora. Evaluation is performed on an internally annotated dataset using standard information retrieval metrics such as normalized discounted cumulative gain (nDCG), precision/recall at n, mean reciprocal rank, and mean average precision (MAP). The results indicate that our hybrid, fine-tuned, domain-adapted search engine is a more effective approach in responding to the information needs of our mortgage officers compared to traditional search techniques. We aim to publish the internally-annotated evaluation and training datasets in the near future.