Diversifying Database Activity Monitoring with Bandits

TL;DR

This paper introduces a novel bandit-based sampling algorithm for database activity monitoring that enhances diversity and coverage without compromising alert quality, addressing limitations of manual policy-based sampling.

Contribution

It redefines the data sampling problem as a multi-armed bandit problem and presents a new algorithm combining expert knowledge with random exploration.

Findings

Adding diversity improves coverage of database activity

Bandit-based sampling maintains alert quality

Enhanced diversity aids downstream event detection

Abstract

Database activity monitoring (DAM) systems are commonly used by organizations to protect the organizational data, knowledge and intellectual properties. In order to protect organizations database DAM systems have two main roles, monitoring (documenting activity) and alerting to anomalous activity. Due to high-velocity streams and operating costs, such systems are restricted to examining only a sample of the activity. Current solutions use policies, manually crafted by experts, to decide which transactions to monitor and log. This limits the diversity of the data collected. Bandit algorithms, which use reward functions as the basis for optimization while adding diversity to the recommended set, have gained increased attention in recommendation systems for improving diversity. In this work, we redefine the data sampling problem as a special case of the multi-armed bandit (MAB) problem and present a novel algorithm, which combines expert knowledge with random exploration. We analyze the effect of diversity on coverage and downstream event detection tasks using a simulated dataset. In doing so, we find that adding diversity to the sampling using the bandit-based approach works well for this task and maximizing population coverage without decreasing the quality in terms of issuing alerts about events.