Processing Unknown Words in HPSG

TL;DR

This paper presents a system for incrementally updating the properties of unknown German words in an HPSG framework, using context-based inference and revisable information to improve lexical entries during parsing.

Contribution

It introduces a novel, uniform approach to handling unknown words in HPSG by modeling unknownness as revisable, information-based, and context-inferred, with an implementation for German.

Findings

Effective incremental lexical updating demonstrated

Revisable information allows flexible lexical refinement

System successfully infers properties of unknown words

Abstract

The lexical acquisition system presented in this paper incrementally updates linguistic properties of unknown words inferred from their surrounding context by parsing sentences with an HPSG grammar for German. We employ a gradual, information-based concept of ``unknownness'' providing a uniform treatment for the range of completely known to maximally unknown lexical entries. ``Unknown'' information is viewed as revisable information, which is either generalizable or specializable. Updating takes place after parsing, which only requires a modified lexical lookup. Revisable pieces of information are identified by grammar-specified declarations which provide access paths into the parse feature structure. The updating mechanism revises the corresponding places in the lexical feature structures iff the context actually provides new information. For revising generalizable information, type union is required. A worked-out example demonstrates the inferential capacity of our implemented system.