Learning Discriminative Relational Features for Sequence Labeling

TL;DR

This paper introduces a novel hierarchical kernel learning approach for optimally discovering discriminative relational features in sequence labeling, addressing the exponential search space challenge and improving model accuracy.

Contribution

It proposes the StructHKL method for structured output spaces and two solutions for learning complex relational features, enhancing feature discovery in sequence labeling.

Findings

StructHKL efficiently explores hierarchical feature space.

Relational kernels implicitly compute similarity for complex features.

Empirical results show effectiveness in specific settings.

Abstract

Discovering relational structure between input features in sequence labeling models has shown to improve their accuracy in several problem settings. However, the search space of relational features is exponential in the number of basic input features. Consequently, approaches that learn relational features, tend to follow a greedy search strategy. In this paper, we study the possibility of optimally learning and applying discriminative relational features for sequence labeling. For learning features derived from inputs at a particular sequence position, we propose a Hierarchical Kernels-based approach (referred to as Hierarchical Kernel Learning for Structured Output Spaces - StructHKL). This approach optimally and efficiently explores the hierarchical structure of the feature space for problems with structured output spaces such as sequence labeling. Since the StructHKL approach has limitations in learning complex relational features derived from inputs at relative positions, we propose two solutions to learn relational features namely, (i) enumerating simple component features of complex relational features and discovering their compositions using StructHKL and (ii) leveraging relational kernels, that compute the similarity between instances implicitly, in the sequence labeling problem. We perform extensive empirical evaluation on publicly available datasets and record our observations on settings in which certain approaches are effective.