'Layer su Layer': Identifying and Disambiguating the Italian NPN Construction in BERT's family
Greta Gorzoni, Ludovica Pannitto, Francesca Masini

TL;DR
This paper investigates how BERT's internal layers encode the Italian NPN construction, providing empirical insights into the relationship between linguistic theory and neural language models.
Contribution
It extends probing methods to a lesser-studied language, challenging assumptions and analyzing BERT's encoding of constructional form and meaning.
Findings
BERT's layers encode the Italian NPN construction to varying degrees.
Layer-wise probing reveals specific layers reflect constructional form and meaning.
Results inform the dialogue between linguistic theory and neural models.
Abstract
Interpretability research has highlighted the importance of evaluating Pretrained Language Models (PLMs) and in particular contextual embeddings against explicit linguistic theories to determine what linguistic information they encode. This study focuses on the Italian NPN (noun-preposition-noun) constructional family, challenging some of the theoretical and methodological assumptions underlying previous experimental designs and extending this type of research to a lesser-investigated language. Contextual vector representations are extracted from BERT and used as input to layer-wise probing classifiers, systematically evaluating information encoded across the model's internal layers. The results shed light on the extent to which constructional form and meaning are reflected in contextual embeddings, contributing empirical evidence to the dialogue between constructionist theory and…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
