Detecting Anomalies Through Contrast in Heterogeneous Data

TL;DR

This paper introduces a novel contrastive learning model for anomaly detection in heterogeneous trade data, effectively identifying illegal timber trade activities without requiring labeled data.

Contribution

The paper proposes a new unsupervised anomaly detection model using contrastive learning with an asymmetric autoencoder tailored for heterogeneous data.

Findings

Effective detection of anomalies in timber trade data

Robustness to hyper-parameter changes

Handles large arity categorical variables

Abstract

Detecting anomalies has been a fundamental approach in detecting potentially fraudulent activities. Tasked with detection of illegal timber trade that threatens ecosystems and economies and association with other illegal activities, we formulate our problem as one of anomaly detection. Among other challenges annotations are unavailable for our large-scale trade data with heterogeneous features (categorical and continuous), that can assist in building automated systems to detect fraudulent transactions. Modelling the task as unsupervised anomaly detection, we propose a novel model Contrastive Learning based Heterogeneous Anomaly Detector to address shortcomings of prior models. Our model uses an asymmetric autoencoder that can effectively handle large arity categorical variables, but avoids assumptions about structure of data in low-dimensional latent space and is robust to changes to hyper-parameters. The likelihood of data is approximated through an estimator network, which is jointly trained with the autoencoder,using negative sampling. Further the details and intuition for an effective negative sample generation approach for heterogeneous data are outlined. We provide a qualitative study to showcase the effectiveness of our model in detecting anomalies in timber trade.