Integrating Transparent Models, LLMs, and Practitioner-in-the-Loop: A Case of Nonprofit Program Evaluation

TL;DR

This paper presents a practitioner-in-the-loop workflow combining transparent decision-tree models with large language models to enhance interpretability, accuracy, and practical insights in nonprofit program evaluation.

Contribution

It introduces a novel workflow integrating transparent models, LLMs, and practitioner input for improved case-level evaluation in nonprofit settings.

Findings

Enhanced predictive accuracy and interpretability.

Practitioner involvement improves model relevance.

Generated insights are actionable and trustworthy.

Abstract

Public and nonprofit organizations often hesitate to adopt AI tools because most models are opaque even though standard approaches typically analyze aggregate patterns rather than offering actionable, case-level guidance. This study tests a practitioner-in-the-loop workflow that pairs transparent decision-tree models with large language models (LLMs) to improve predictive accuracy, interpretability, and the generation of practical insights. Using data from an ongoing college-success program, we build interpretable decision trees to surface key predictors. We then provide each tree's structure to an LLM, enabling it to reproduce case-level predictions grounded in the transparent models. Practitioners participate throughout feature engineering, model design, explanation review, and usability assessment, ensuring that field expertise informs the analysis at every stage. Results show that integrating transparent models, LLMs, and practitioner input yields accurate, trustworthy, and actionable case-level evaluations, offering a viable pathway for responsible AI adoption in the public and nonprofit sectors.