OPTIAGENT: A Physics-Driven Agentic Framework for Automated Optical Design

TL;DR

OPTIAGENT introduces a physics-driven, LLM-based framework that automates optical lens design, enabling non-experts to develop high-quality optical systems through a hybrid learning and optimization approach.

Contribution

This work pioneers the integration of large language models with physics-based optimization for automated optical design, bridging expertise gaps and improving design success.

Findings

Outperforms traditional optimization algorithms in lens design quality.

Enables non-experts to create functional optical systems.

Demonstrates the effectiveness of hybrid LLM and physics-based training.

Abstract

Optical design is the process of configuring optical elements to precisely manipulate light for high-fidelity imaging. It is inherently a highly non-convex optimization problem that relies heavily on human heuristic expertise and domain-specific knowledge. While Large Language Models (LLMs) possess extensive optical knowledge, their capabilities in leveraging the knowledge in designing lens system remain significantly constrained. This work represents the first attempt to employ LLMs in the field of optical design. We bridge the expertise gap by enabling users without formal optical training to successfully develop functional lens systems. Concretely, we curate a comprehensive dataset, named OptiDesignQA, which encompasses both classical lens systems sourced from standard optical textbooks and novel configurations generated by automated design algorithms for training and evaluation. Furthermore, we inject domain-specific optical expertise into the LLM through a hybrid objective of full-system synthesis and lens completion. To align the model with optical principles, we employ Group Relative Policy Optimization Done Right (DrGRPO) guided by Optical Lexicographic Reward for physics-driven policy alignment. This reward system incorporates structural format rewards, physical feasibility rewards, light-manipulation accuracy, and LLM-based heuristics. Finally, our model integrates with specialized optical optimization routines for end-to-end fine-tuning and precision refinement. We benchmark our proposed method against both traditional optimization-based automated design algorithms and LLM counterparts, and experimental results show the superiority of our method.