Expert Knowledge-Guided Length-Variant Hierarchical Label Generation for Proposal Classification

TL;DR

This paper introduces a deep hierarchical multi-label classification framework for research proposals, leveraging expert partial labels and textual data to improve classification accuracy in a complex, length-variant label space.

Contribution

It proposes a novel deep learning model that jointly utilizes partial expert labels, hierarchical structure, and textual information for proposal classification.

Findings

Outperforms existing methods in proposal classification accuracy

Effectively integrates partial labels and textual data

Automatically determines optimal label sequence length

Abstract

To advance the development of science and technology, research proposals are submitted to open-court competitive programs developed by government agencies (e.g., NSF). Proposal classification is one of the most important tasks to achieve effective and fair review assignments. Proposal classification aims to classify a proposal into a length-variant sequence of labels. In this paper, we formulate the proposal classification problem into a hierarchical multi-label classification task. Although there are certain prior studies, proposal classification exhibit unique features: 1) the classification result of a proposal is in a hierarchical discipline structure with different levels of granularity; 2) proposals contain multiple types of documents; 3) domain experts can empirically provide partial labels that can be leveraged to improve task performances. In this paper, we focus on developing a new deep proposal classification framework to jointly model the three features. In particular, to sequentially generate labels, we leverage previously-generated labels to predict the label of next level; to integrate partial labels from experts, we use the embedding of these empirical partial labels to initialize the state of neural networks. Our model can automatically identify the best length of label sequence to stop next label prediction. Finally, we present extensive results to demonstrate that our method can jointly model partial labels, textual information, and semantic dependencies in label sequences, and, thus, achieve advanced performances.