Classifying Sequences of Extreme Length with Constant Memory Applied to Malware Detection

TL;DR

This paper introduces a memory-efficient approach to sequence classification that handles extremely long inputs, enabling improved malware detection with larger datasets and more complex models.

Contribution

The authors develop a novel memory-invariant max pooling method and enhance MalConv with a global channel gating attention mechanism for processing 100 million time steps.

Findings

Memory usage is reduced by 116 times, enabling processing of longer sequences.

Training time is reduced by up to 25.8 times on original datasets.

The new architecture improves feature interaction learning across extremely long sequences.

Abstract

Recent works within machine learning have been tackling inputs of ever-increasing size, with cybersecurity presenting sequence classification problems of particularly extreme lengths. In the case of Windows executable malware detection, inputs may exceed $100$ MB, which corresponds to a time series with $T=100,000,000$ steps. To date, the closest approach to handling such a task is MalConv, a convolutional neural network capable of processing up to $T=2,000,000$ steps. The $\mathcal{O}(T)$ memory of CNNs has prevented further application of CNNs to malware. In this work, we develop a new approach to temporal max pooling that makes the required memory invariant to the sequence length $T$. This makes MalConv $116\times$ more memory efficient, and up to $25.8\times$ faster to train on its original dataset, while removing the input length restrictions to MalConv. We re-invest these gains into improving the MalConv architecture by developing a new Global Channel Gating design, giving us an attention mechanism capable of learning feature interactions across 100 million time steps in an efficient manner, a capability lacked by the original MalConv CNN. Our implementation can be found at https://github.com/NeuromorphicComputationResearchProgram/MalConv2