Machine Learning of User Profiles: Representational Issues

TL;DR

This paper explores how to create understandable and accurate user profiles for information retrieval by using hierarchical representations and combining NLP with machine learning, demonstrated through experiments on online newspaper content.

Contribution

It highlights the significance of a suitable generalization hierarchy and shows the effectiveness of combining NLP techniques with ML for user profiling.

Findings

Hierarchical representations improve profile accuracy

Combining NLP with ML enhances profile comprehensibility

Subject features from thesauri are effective in profiling

Abstract

As more information becomes available electronically, tools for finding information of interest to users becomes increasingly important. The goal of the research described here is to build a system for generating comprehensible user profiles that accurately capture user interest with minimum user interaction. The research described here focuses on the importance of a suitable generalization hierarchy and representation for learning profiles which are predictively accurate and comprehensible. In our experiments we evaluated both traditional features based on weighted term vectors as well as subject features corresponding to categories which could be drawn from a thesaurus. Our experiments, conducted in the context of a content-based profiling system for on-line newspapers on the World Wide Web (the IDD News Browser), demonstrate the importance of a generalization hierarchy and the promise of combining natural language processing techniques with machine learning (ML) to address an information retrieval (IR) problem.