Weakly-supervised Dictionary Learning

TL;DR

This paper introduces a probabilistic framework for weakly-supervised analysis dictionary learning tailored for time-series data, enhancing classification by incorporating label information and sparsity constraints.

Contribution

It proposes a novel discriminative probabilistic model and efficient EM algorithm for weakly-supervised analysis dictionary learning, with reformulated graphical models for computational efficiency.

Findings

Demonstrates improved classification on synthetic data

Validates effectiveness on real-world time-series data

Introduces efficient EM algorithm with chain and tree reformulations

Abstract

We present a probabilistic modeling and inference framework for discriminative analysis dictionary learning under a weak supervision setting. Dictionary learning approaches have been widely used for tasks such as low-level signal denoising and restoration as well as high-level classification tasks, which can be applied to audio and image analysis. Synthesis dictionary learning aims at jointly learning a dictionary and corresponding sparse coefficients to provide accurate data representation. This approach is useful for denoising and signal restoration, but may lead to sub-optimal classification performance. By contrast, analysis dictionary learning provides a transform that maps data to a sparse discriminative representation suitable for classification. We consider the problem of analysis dictionary learning for time-series data under a weak supervision setting in which signals are assigned with a global label instead of an instantaneous label signal. We propose a discriminative probabilistic model that incorporates both label information and sparsity constraints on the underlying latent instantaneous label signal using cardinality control. We present the expectation maximization (EM) procedure for maximum likelihood estimation (MLE) of the proposed model. To facilitate a computationally efficient E-step, we propose both a chain and a novel tree graph reformulation of the graphical model. The performance of the proposed model is demonstrated on both synthetic and real-world data.