# OPERAS decision support system versus manual job coding: a quantitative analysis on coding time and inter-coder reliability

## TL;DR

A decision support system called OPERAS improves job coding speed and consistency compared to manual coding by experts.

## Contribution

This study empirically demonstrates that the OPERAS system improves coding efficiency and reliability in occupational classification.

## Key findings

- OPERAS reduced median coding time from 60.8 seconds to 38 seconds.
- Inter-coder reliability was higher with OPERAS than manual coding for both occupation and activity sector classifications.
- OPERAS received a System Usability Scale score of 75.5, indicating good usability.

## Abstract

The manual coding of job descriptions is time-consuming, expensive and requires expert knowledge. Decision support systems (DSS) provide a valuable alternative by offering automated suggestions that support decision-making, improving efficiency while allowing manual corrections to ensure reliability. However, this claim has not been proven with expert coders. This study aims to fill this omission by comparing manual with decision-supported coding, using the new DSS OPERAS.

Five expert coders proficient in using the French classification systems for occupations PCS2003 and activity sectors NAF2008 each successively coded two subsets of job descriptions from the CONSTANCES cohort manually and using OPERAS. Subsequently, we assessed coding time and inter-coder reliability of assigning occupation and activity sector codes while accounting for individual differences and the perceived usability of OPERAS, measured using the System Usability Scale (SUS; range 0–100).

OPERAS usage substantially outperformed manual coding for all coders on both coding time and inter-coder reliability. The median job description coding time was 38 s using OPERAS versus 60.8 s while manually coding. Inter-coder reliability (in Cohen’s kappa) ranged 0.61–0.70 and 0.56–0.61 for the PCS, while ranging 0.38–0.61 and 0.34–0.61 for the NAF for OPERAS and manual coding, respectively. The average SUS score was 75.5, indicating good usability.

Compared with manual coding, using OPERAS as DSS for occupational coding improved coding time and inter-coder reliability. Subsequent comparison studies could use OPERAS’ ISCO-88 and ISCO-68 classification models. Consequently, OPERAS facilitates large, harmonised job coding in large-scale occupational health research.