Utilizing BERT for Information Retrieval: Survey, Applications, Resources, and Challenges
Jiajia Wang, Jimmy X. Huang, Xinhui Tu, Junmei Wang, Angela J. Huang,, Md Tahmid Rahman Laskar, Amran Bhuiyan

TL;DR
This survey reviews BERT-based methods for information retrieval, compares them with large language models like ChatGPT, and discusses resources, challenges, and future research directions in applying BERT to IR tasks.
Contribution
It provides a comprehensive analysis of BERT-based IR techniques, categorizes approaches, compares with LLMs, and offers resources and future research insights.
Findings
BERT-based models outperform some LLMs in specific IR tasks.
Fine-tuned BERT encoders are more cost-effective than generative LLMs.
The survey categorizes IR approaches into six high-level groups.
Abstract
Recent years have witnessed a substantial increase in the use of deep learning to solve various natural language processing (NLP) problems. Early deep learning models were constrained by their sequential or unidirectional nature, such that they struggled to capture the contextual relationships across text inputs. The introduction of bidirectional encoder representations from transformers (BERT) leads to a robust encoder for the transformer model that can understand the broader context and deliver state-of-the-art performance across various NLP tasks. This has inspired researchers and practitioners to apply BERT to practical problems, such as information retrieval (IR). A survey that focuses on a comprehensive analysis of prevalent approaches that apply pretrained transformer encoders like BERT to IR can thus be useful for academia and the industry. In light of this, we revisit a variety…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsWeb Data Mining and Analysis · Service-Oriented Architecture and Web Services · Text and Document Classification Technologies
MethodsRefunds@Expedia|||How do I get a full refund from Expedia? · Attention Is All You Need · Linear Layer · WordPiece · Layer Normalization · Dropout · Multi-Head Attention · Attention Dropout · Linear Warmup With Linear Decay · Softmax
