Construction Grammar and Language Models
Harish Tayyar Madabushi, Laurence Romain, Petar Milin, Dagmar, Divjak

TL;DR
This paper explores how deep learning language models can access and represent constructional knowledge, highlighting opportunities for collaboration between NLP and Construction Grammar to enhance linguistic understanding.
Contribution
It provides a comprehensive overview of computational approaches, especially deep learning models, and investigates their potential to capture constructional information in language.
Findings
Deep learning models show evidence of constructional knowledge emergence.
Construction Grammar can inform the development of more linguistically aware models.
Challenges remain in modeling certain aspects of Construction Grammar with current models.
Abstract
Recent progress in deep learning and natural language processing has given rise to powerful models that are primarily trained on a cloze-like task and show some evidence of having access to substantial linguistic information, including some constructional knowledge. This groundbreaking discovery presents an exciting opportunity for a synergistic relationship between computational methods and Construction Grammar research. In this chapter, we explore three distinct approaches to the interplay between computational methods and Construction Grammar: (i) computational methods for text analysis, (ii) computational Construction Grammar, and (iii) deep learning models, with a particular focus on language models. We touch upon the first two approaches as a contextual foundation for the use of computational methods before providing an accessible, yet comprehensive overview of deep learning…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsNatural Language Processing Techniques · Topic Modeling · Speech and dialogue systems
MethodsFocus
