Research Community Perspectives on "Intelligence" and Large Language Models

TL;DR

This paper surveys researchers across multiple fields to understand their definitions of 'intelligence' in NLP, revealing common criteria and attitudes towards current systems' intelligence and research goals.

Contribution

It provides empirical insights into community perspectives on 'intelligence' and clarifies the criteria and perceptions influencing NLP research directions.

Findings

Community agrees on generalization, adaptability, reasoning as key intelligence criteria.

Only 29% see current NLP systems as 'intelligent'.

16.2% prioritize developing intelligent systems as a research goal.

Abstract

Despite the widespread use of ''artificial intelligence'' (AI) framing in Natural Language Processing (NLP) research, it is not clear what researchers mean by ''intelligence''. To that end, we present the results of a survey on the notion of ''intelligence'' among researchers and its role in the research agenda. The survey elicited complete responses from 303 researchers from a variety of fields including NLP, Machine Learning (ML), Cognitive Science, Linguistics, and Neuroscience. We identify 3 criteria of intelligence that the community agrees on the most: generalization, adaptability, & reasoning. Our results suggests that the perception of the current NLP systems as ''intelligent'' is a minority position (29%). Furthermore, only 16.2% of the respondents see developing intelligent systems as a research goal, and these respondents are more likely to consider the current systems intelligent.