Variational Bayes survival analysis for unemployment modelling

TL;DR

This paper introduces a novel variational Bayes neural network model for survival analysis of unemployment data, effectively handling high-cardinality categorical features and censored records to predict employment probabilities over time.

Contribution

It presents a new deep neural network-based survival analysis model using variational Bayes for parameter estimation, capable of analyzing complex categorical data efficiently.

Findings

Model accurately predicts employment probabilities over time.

Effective handling of high-cardinality categorical features.

Applicable to other censored, multi-dimensional data domains.

Abstract

Mathematical modelling of unemployment dynamics attempts to predict the probability of a job seeker finding a job as a function of time. This is typically achieved by using information in unemployment records. These records are right censored, making survival analysis a suitable approach for parameter estimation. The proposed model uses a deep artificial neural network (ANN) as a non-linear hazard function. Through embedding, high-cardinality categorical features are analysed efficiently. The posterior distribution of the ANN parameters are estimated using a variational Bayes method. The model is evaluated on a time-to-employment data set spanning from 2011 to 2020 provided by the Slovenian public employment service. It is used to determine the employment probability over time for each individual on the record. Similar models could be applied to other questions with multi-dimensional, high-cardinality categorical data including censored records. Such data is often encountered in personal records, for example in medical records.