Relevance Scoring of Triples Using Ordinal Logistic Classification - The Celosia Triple Scorer at WSDM Cup 2017

TL;DR

This paper presents a supervised machine learning system that predicts relevance scores for triples related to profession and nationality using ordinal logistic regression, achieving competitive accuracy and correlation scores.

Contribution

The paper introduces an ordinal logistic regression approach with feature engineering for triple relevance scoring in WSDM Cup 2017.

Findings

Achieved 0.73 accuracy in relevance prediction

Attained 0.36 Kendall's tau correlation score

Demonstrated effectiveness of ordinal logistic regression for triple scoring

Abstract

In this paper, we report our participation in the Task 2: Triple Scoring of WSDM Cup challenge 2017. In this task, we were provided with triples of "type-like" relations which were given human-annotated relevance scores ranging from 0 to 7, with 7 being the "most relevant" and 0 being the "least relevant". The task focuses on two such relations: profession and nationality. We built a system which could automatically predict the relevance scores for unseen triples. Our model is primarily a supervised machine learning based one in which we use well-designed features which are used to a make a Logistic Ordinal Regression based classification model. The proposed system achieves an overall accuracy score of 0.73 and Kendall's tau score of 0.36.