Hierarchical Knowledge Distillation on Text Graph for Data-limited Attribute Inference

TL;DR

This paper introduces a hierarchical knowledge distillation approach on text graphs to improve user attribute inference from social media data, especially effective in few-shot scenarios with limited labeled texts.

Contribution

It proposes a novel hierarchical knowledge distillation method on refined text graphs for better attribute inference in data-scarce social media contexts.

Findings

Achieves state-of-the-art performance on social media datasets.

Significantly reduces the need for labeled data.

Enhances model generalization with cross-domain and unlabeled texts.

Abstract

The popularization of social media increases user engagements and generates a large amount of user-oriented data. Among them, text data (e.g., tweets, blogs) significantly attracts researchers and speculators to infer user attributes (e.g., age, gender, location) for fulfilling their intents. Generally, this line of work casts attribute inference as a text classification problem, and starts to leverage graph neural networks (GNNs) to utilize higher-level representations of source texts. However, these text graphs are constructed over words, suffering from high memory consumption and ineffectiveness on few labeled texts. To address this challenge, we design a text-graph-based few-shot learning model for attribute inferences on social media text data. Our model first constructs and refines a text graph using manifold learning and message passing, which offers a better trade-off between expressiveness and complexity. Afterwards, to further use cross-domain texts and unlabeled texts to improve few-shot performance, a hierarchical knowledge distillation is devised over text graph to optimize the problem, which derives better text representations, and advances model generalization ability. Experiments on social media datasets demonstrate the state-of-the-art performance of our model on attribute inferences with considerably fewer labeled texts.