Harnessing label semantics to extract higher performance under noisy label for Company to Industry matching

TL;DR

This paper presents a novel ML pipeline that leverages label semantics and similarity matching to improve industry tag assignment for companies, especially under noisy labeling conditions, enhancing robustness and efficiency.

Contribution

It introduces a semantic similarity-based approach with a Label Similarity Matrix and minimum labeling strategy to handle noisy labels in company-industry classification.

Findings

Significant improvement in robustness against noisy labels.

Enhanced predictive accuracy over traditional methods.

Effective use of semantic similarity for multi-label classification.

Abstract

Assigning appropriate industry tag(s) to a company is a critical task in a financial institution as it impacts various financial machineries. Yet, it remains a complex task. Typically, such industry tags are to be assigned by Subject Matter Experts (SME) after evaluating company business lines against the industry definitions. It becomes even more challenging as companies continue to add new businesses and newer industry definitions are formed. Given the periodicity of the task it is reasonable to assume that an Artificial Intelligent (AI) agent could be developed to carry it out in an efficient manner. While this is an exciting prospect, the challenges appear from the need of historical patterns of such tag assignments (or Labeling). Labeling is often considered the most expensive task in Machine Learning (ML) due its dependency on SMEs and manual efforts. Therefore, often, in enterprise set up, an ML project encounters noisy and dependent labels. Such labels create technical hindrances for ML Models to produce robust tag assignments. We propose an ML pipeline which uses semantic similarity matching as an alternative to multi label text classification, while making use of a Label Similarity Matrix and a minimum labeling strategy. We demonstrate this pipeline achieves significant improvements over the noise and exhibit robust predictive capabilities.