DiSciPLE: Learning Interpretable Programs for Scientific Visual Discovery

TL;DR

DiSciPLE is an evolutionary algorithm that uses large language models to automatically generate interpretable Python programs explaining visual scientific data, achieving state-of-the-art results in real-world tasks.

Contribution

It introduces DiSciPLE, a novel method combining LLMs and evolutionary algorithms to learn interpretable programs for scientific visual data analysis.

Findings

Achieves 35% lower error in population density estimation.

Learns state-of-the-art interpretable programs on new tasks.

Incorporates program critic and simplifier for improved synthesis.

Abstract

Visual data is used in numerous different scientific workflows ranging from remote sensing to ecology. As the amount of observation data increases, the challenge is not just to make accurate predictions but also to understand the underlying mechanisms for those predictions. Good interpretation is important in scientific workflows, as it allows for better decision-making by providing insights into the data. This paper introduces an automatic way of obtaining such interpretable-by-design models, by learning programs that interleave neural networks. We propose DiSciPLE (Discovering Scientific Programs using LLMs and Evolution) an evolutionary algorithm that leverages common sense and prior knowledge of large language models (LLMs) to create Python programs explaining visual data. Additionally, we propose two improvements: a program critic and a program simplifier to improve our method further to synthesize good programs. On three different real-world problems, DiSciPLE learns state-of-the-art programs on novel tasks with no prior literature. For example, we can learn programs with 35% lower error than the closest non-interpretable baseline for population density estimation.