How Distributed are Distributed Representations? An Observation on the Locality of Syntactic Information in Verb Agreement Tasks Article Swipe
Related Concepts
Computer science
Natural language processing
Participle
Artificial intelligence
Sentence
Locality
Agreement
Transformer
Representation (politics)
Linguistics
Verb
Syntactic predicate
Grammar
Selection (genetic algorithm)
Object (grammar)
Parsing
Physics
Political science
Law
Voltage
Quantum mechanics
Philosophy
Politics
Bingzhi Li
,
Guillaume Wisniewski
,
Benoît Crabbé
·
YOU?
·
· 2022
· Open Access
·
· DOI: https://doi.org/10.18653/v1/2022.acl-short.54
· OA: W4285220062
YOU?
·
· 2022
· Open Access
·
· DOI: https://doi.org/10.18653/v1/2022.acl-short.54
· OA: W4285220062
This work addresses the question of the localization of syntactic information encoded in the transformers representations. We tackle this question from two perspectives, considering the object-past participle agreement in French, by identifying, first, in which part of the sentence and, second, in which part of the representation the syntactic information is encoded. The results of our experiments, using probing, causal analysis and feature selection method, show that syntactic information is encoded locally in a way consistent with the French grammar.
Related Topics
Finding more related topics…