Text Production and Comprehension by Human and Artificial Intelligence: Interdisciplinary Workshop Report

TL;DR

This interdisciplinary workshop report explores how large language models relate to human cognition in text comprehension and production, highlighting insights, limitations, and future research directions for AI-human collaboration.

Contribution

It synthesizes interdisciplinary perspectives to understand the relationship between AI language models and human cognitive processes in text tasks.

Findings

LLMs can provide insights into human language processing

Fine-tuning LLMs with human feedback improves alignment with human cognition

Human-AI collaboration presents both opportunities and challenges in language tasks

Abstract

This report synthesizes the outcomes of a recent interdisciplinary workshop that brought together leading experts in cognitive psychology, language learning, and artificial intelligence (AI)-based natural language processing (NLP). The workshop, funded by the National Science Foundation, aimed to address a critical knowledge gap in our understanding of the relationship between AI language models and human cognitive processes in text comprehension and composition. Through collaborative dialogue across cognitive, linguistic, and technological perspectives, workshop participants examined the underlying processes involved when humans produce and comprehend text, and how AI can both inform our understanding of these processes and augment human capabilities. The workshop revealed emerging patterns in the relationship between large language models (LLMs) and human cognition, with highlights on both the capabilities of LLMs and their limitations in fully replicating human-like language understanding and generation. Key findings include the potential of LLMs to offer insights into human language processing, the increasing alignment between LLM behavior and human language processing when models are fine-tuned with human feedback, and the opportunities and challenges presented by human-AI collaboration in language tasks. By synthesizing these findings, this report aims to guide future research, development, and implementation of LLMs in cognitive psychology, linguistics, and education. It emphasizes the importance of ethical considerations and responsible use of AI technologies while striving to enhance human capabilities in text comprehension and production through effective human-AI collaboration.