Building a Structured Query Engine

TL;DR

This paper presents a new structured query engine supporting real-time data indexing and retrieval via RESTful API, enabling complex pattern searches and demonstrated through a movie recommendation system.

Contribution

Developed a structured query engine with efficient data structures supporting real-time indexing and complex queries, similar to ElasticSearch.

Findings

Real-time data indexing and retrieval achieved

Supports complex structured queries efficiently

Demonstrated application in a movie recommendation system

Abstract

Finding patterns in data and being able to retrieve information from those patterns is an important task in Information retrieval. Complex search requirements which are not fulfilled by simple string matching and require exploring certain patterns in data demand a better query engine that can support searching via structured queries. In this article, we built a structured query engine which supports searching data through structured queries on the lines of ElasticSearch. We will show how we achieved real time indexing and retrieving of data through a RESTful API and how complex queries can be created and processed using efficient data structures we created for storing the data in structured way. Finally, we will conclude with an example of movie recommendation system built on top of this query engine.