Opportunistically Parallel Lambda Calculus

TL;DR

This paper introduces Opal, a lambda calculus-based scripting language that automatically executes independent external calls in parallel, significantly reducing total runtime and latency for scripts involving heavy external computations like large language models.

Contribution

It presents a novel opportunistic evaluation strategy for scripting languages that ensures confluence and preserves programmer intent, with an implementation demonstrating substantial performance gains.

Findings

Up to 6.2x faster total runtime compared to standard Python

Up to 12.7x reduced latency in external calls

Close performance to manually optimized asynchronous Rust

Abstract

Scripting languages are widely used to compose external calls such as native libraries and network services. In such scripts, execution time is often dominated by waiting for these external calls, rendering traditional single-language optimizations ineffective. To address this, we propose a novel opportunistic evaluation strategy for scripting languages based on a core lambda calculus that automatically dispatches independent external calls in parallel and streams their results. We prove that our approach is confluent, ensuring that it preserves the programmer's original intent, and that it eventually executes every external call. We implement this approach in a scripting language called Opal. We demonstrate the versatility and performance of Opal, focusing on programs that invoke heavy external computation through the use of large language models (LLMs) and other APIs. Across five scripts, we compare to several state-of-the-art baselines and show that opportunistic evaluation improves total running time (up to $6.2\times$) and latency (up to $12.7\times$) compared to standard sequential Python, while performing very close (between $1.3\%$ and $18.5\%$ running time overhead) to hand-tuned manually optimized asynchronous Rust. For Tree-of-Thoughts, a prominent LLM reasoning approach, we achieve a $6.2\times$ performance improvement over the authors' own implementation.