Unified Semantic Typing with Meaningful Label Inference

TL;DR

UniST is a unified semantic typing framework that embeds inputs and labels into a shared space, enabling flexible, multi-task learning and improved generalization across entity, relation, and event classification tasks.

Contribution

This paper introduces UniST, a novel unified semantic typing model that incorporates task descriptions and joint embeddings to handle multiple tasks without task-specific components.

Findings

Achieves strong performance on entity, relation, and event typing tasks.

Effectively transfers semantic knowledge to rarely seen and unseen types.

Supports multi-task training with comparable or better results than single-task models.

Abstract

Semantic typing aims at classifying tokens or spans of interest in a textual context into semantic categories such as relations, entity types, and event types. The inferred labels of semantic categories meaningfully interpret how machines understand components of text. In this paper, we present UniST, a unified framework for semantic typing that captures label semantics by projecting both inputs and labels into a joint semantic embedding space. To formulate different lexical and relational semantic typing tasks as a unified task, we incorporate task descriptions to be jointly encoded with the input, allowing UniST to be adapted to different tasks without introducing task-specific model components. UniST optimizes a margin ranking loss such that the semantic relatedness of the input and labels is reflected from their embedding similarity. Our experiments demonstrate that UniST achieves strong performance across three semantic typing tasks: entity typing, relation classification and event typing. Meanwhile, UniST effectively transfers semantic knowledge of labels and substantially improves generalizability on inferring rarely seen and unseen types. In addition, multiple semantic typing tasks can be jointly trained within the unified framework, leading to a single compact multi-tasking model that performs comparably to dedicated single-task models, while offering even better transferability.