# Supervised Negative Binomial Classifier for Probabilistic Record Linkage

## TL;DR

This paper introduces a supervised probabilistic classifier based on a mixture of Poisson distributions with latent variables, designed for effective record linkage across diverse, sparse, and streaming datasets, leveraging Bayesian techniques and gamma priors.

## Contribution

It presents a novel graphical model classifier that incorporates gamma priors and supervised labels for probabilistic record linkage, capable of handling sparse and streaming data.

## Key findings

- Effective in linking records across diverse datasets
- Handles sparsity and streaming data efficiently
- Utilizes Bayesian modeling with gamma priors for improved accuracy

## Abstract

Motivated by the need of the linking records across various databases, we propose a novel graphical model based classifier that uses a mixture of Poisson distributions with latent variables. The idea is to derive insight into each pair of hypothesis records that match by inferring its underlying latent rate of error using Bayesian Modeling techniques. The novel approach of using gamma priors for learning the latent variables along with supervised labels is unique and allows for active learning. The naive assumption is made deliberately as to the independence of the fields to propose a generalized theory for this class of problems and not to undermine the hierarchical dependencies that could be present in different scenarios. This classifier is able to work with sparse and streaming data. The application to record linkage is able to meet several challenges of sparsity, data streams and varying nature of the data-sets.