Online Transition-Based Feature Generation for Anomaly Detection in Concurrent Data Streams

TL;DR

This paper presents TFGen, an online transition-based feature generator that efficiently creates encoded features from concurrent activity data streams, enhancing anomaly detection across various domains.

Contribution

The paper introduces TFGen, a novel online feature generation method capable of handling concurrent data streams for improved anomaly detection.

Findings

Efficient online processing of concurrent data streams.

Effective encoding of historical activity data.

Applicable across multiple activity data types.

Abstract

In this paper, we introduce the transition-based feature generator (TFGen) technique, which reads general activity data with attributes and generates step-by-step generated data. The activity data may consist of network activity from packets, system calls from processes or classified activity from surveillance cameras. TFGen processes data online and will generate data with encoded historical data for each incoming activity with high computational efficiency. The input activities may concurrently originate from distinct traces or channels. The technique aims to address issues such as domain-independent applicability, the ability to discover global process structures, the encoding of time-series data, and online processing capability.