Latent Feature Mining for Predictive Model Enhancement with Large Language Models

TL;DR

This paper introduces FLAME, a framework using large language models to infer latent features from limited data, improving predictive models in sensitive domains like criminal justice and healthcare.

Contribution

The paper presents a novel text-to-text logical reasoning approach with LLMs to extract latent features, enhancing predictive models in data-constrained, sensitive domains.

Findings

Latent features inferred by FLAME align with ground truth labels.

Enhanced predictive accuracy in criminal justice and healthcare tasks.

Framework demonstrates effective transferability across domains.

Abstract

Predictive modeling often faces challenges due to limited data availability and quality, especially in domains where collected features are weakly correlated with outcomes and where additional feature collection is constrained by ethical or practical difficulties. Traditional machine learning (ML) models struggle to incorporate unobserved yet critical factors. In this work, we introduce an effective approach to formulate latent feature mining as text-to-text propositional logical reasoning. We propose FLAME (Faithful Latent Feature Mining for Predictive Model Enhancement), a framework that leverages large language models (LLMs) to augment observed features with latent features and enhance the predictive power of ML models in downstream tasks. Our framework is generalizable across various domains with necessary domain-specific adaptation, as it is designed to incorporate contextual information unique to each area, ensuring effective transfer to different areas facing similar data availability challenges. We validate our framework with two case studies: (1) the criminal justice system, a domain characterized by limited and ethically challenging data collection; (2) the healthcare domain, where patient privacy concerns and the complexity of medical data limit comprehensive feature collection. Our results show that inferred latent features align well with ground truth labels and significantly enhance the downstream classifier.