# QuesNet: A Unified Representation for Heterogeneous Test Questions

## TL;DR

QuesNet is a novel pre-training framework that effectively learns comprehensive representations of heterogeneous test questions by integrating content, images, and domain knowledge, improving question understanding in online learning systems.

## Contribution

The paper introduces QuesNet, a unified pre-training method with a hierarchical approach and a novel language model objective tailored for heterogeneous educational questions.

## Key findings

- QuesNet outperforms existing methods in question understanding tasks.
- It demonstrates strong adaptability across various question-based applications.
- Experimental results validate the effectiveness of the proposed approach.

## Abstract

Understanding learning materials (e.g. test questions) is a crucial issue in online learning systems, which can promote many applications in education domain. Unfortunately, many supervised approaches suffer from the problem of scarce human labeled data, whereas abundant unlabeled resources are highly underutilized. To alleviate this problem, an effective solution is to use pre-trained representations for question understanding. However, existing pre-training methods in NLP area are infeasible to learn test question representations due to several domain-specific characteristics in education. First, questions usually comprise of heterogeneous data including content text, images and side information. Second, there exists both basic linguistic information as well as domain logic and knowledge. To this end, in this paper, we propose a novel pre-training method, namely QuesNet, for comprehensively learning question representations. Specifically, we first design a unified framework to aggregate question information with its heterogeneous inputs into a comprehensive vector. Then we propose a two-level hierarchical pre-training algorithm to learn better understanding of test questions in an unsupervised way. Here, a novel holed language model objective is developed to extract low-level linguistic features, and a domain-oriented objective is proposed to learn high-level logic and knowledge. Moreover, we show that QuesNet has good capability of being fine-tuned in many question-based tasks. We conduct extensive experiments on large-scale real-world question data, where the experimental results clearly demonstrate the effectiveness of QuesNet for question understanding as well as its superior applicability.