A nonparametric framework for inferring orders of categorical data from category-real ordered pairs

TL;DR

This paper introduces a scalable nonparametric framework using Estimation statistics to infer category orders and magnitude differences from category-real data, demonstrated through income and stock market case studies.

Contribution

The paper presents a novel, scalable nonparametric framework for inferring category orders and difference magnitudes, applicable across various research fields.

Findings

Successfully ordered careers by income, revealing income inequality.

Ordered stock sectors by closing prices, showing market dynamics.

Framework is scalable and applicable to large datasets.

Abstract

Given a dataset of careers and incomes, how large a difference of income between any pair of careers would be? Given a dataset of travel time records, how long do we need to spend more when choosing a public transportation mode $A$ instead of $B$ to travel? In this paper, we propose a framework that is able to infer orders of categories as well as magnitudes of difference of real numbers between each pair of categories using Estimation statistics framework. Not only reporting whether an order of categories exists, but our framework also reports the magnitude of difference of each consecutive pairs of categories in the order. In large dataset, our framework is scalable well compared with the existing framework. The proposed framework has been applied to two real-world case studies: 1) ordering careers by incomes based on information of 350,000 households living in Khon Kaen province, Thailand, and 2) ordering sectors by closing prices based on 1060 companies' closing prices of NASDAQ stock markets between years 2000 and 2016. The results of careers ordering show income inequality among different careers. The stock market results illustrate dynamics of sector domination that can change over time. Our approach is able to be applied in any research area that has category-real ordered pairs. Our proposed "Dominant-Distribution Network" provides a novel approach to gain new insight of analyzing category orders. The software of this framework is available for researchers or practitioners within R package: EDOIF.