Synergizing Human Expertise and AI Efficiency with Language Model for Microscopy Operation and Automated Experiment Design

TL;DR

This paper investigates how large language models like ChatGPT4 can assist in microscopy experiment design, programming, and analysis, highlighting their potential and current limitations in scientific workflows.

Contribution

It demonstrates the utility of ChatGPT4 in converting experimental ideas into executable code and analyzing microscopy images, emphasizing the need for domain-specific fine-tuning.

Findings

ChatGPT4 can generate executable code for microscopy APIs.

GPT4 can analyze microscopy images in a general sense.

Limitations exist in GPT4's ability to perform in-depth analysis and complex experimental design.

Abstract

With the advent of large language models (LLMs), in both the open source and proprietary domains, attention is turning to how to exploit such artificial intelligence (AI) systems in assisting complex scientific tasks, such as material synthesis, characterization, analysis and discovery. Here, we explore the utility of LLM, particularly ChatGPT4, in combination with application program interfaces (APIs) in tasks of experimental design, programming workflows, and data analysis in scanning probe microscopy, using both in-house developed API and API given by a commercial vendor for instrument control. We find that the LLM can be especially useful in converting ideations of experimental workflows to executable code on microscope APIs. Beyond code generation, we find that the GPT4 is capable of analyzing microscopy images in a generic sense. At the same time, we find that GPT4 suffers from inability to extend beyond basic analyses or more in-depth technical experimental design. We argue that a LLM specifically fine-tuned for individual scientific domains can potentially be a better language interface for converting scientific ideations from human experts to executable workflows, such a synergy between human expertise and LLM efficiency in experimentation can open new door for accelerating scientific research, enabling effective experimental protocols archive and sharing in scientific community.