Afra Alishahi
YOU?
Author Swipe
View article: A Linguistically Motivated Analysis of Intonational Phrasing in Text-to-Speech Systems: Revealing Gaps in Syntactic Sensitivity
A Linguistically Motivated Analysis of Intonational Phrasing in Text-to-Speech Systems: Revealing Gaps in Syntactic Sensitivity Open
We analyze the syntactic sensitivity of Text-to-Speech (TTS) systems using methods inspired by psycholinguistic research. Specifically, we focus on the generation of intonational phrase boundaries, which can often be predicted by identifyi…
View article: Using computational modeling to validate the onset of productive determiner–noun combinations in English-learning children
Using computational modeling to validate the onset of productive determiner–noun combinations in English-learning children Open
Language is a productive system––we routinely produce well-formed utterances that we have never heard before. It is, however, difficult to assess when children first achieve linguistic productivity simply because we rarely know all the utt…
View article: How Language Models Prioritize Contextual Grammatical Cues?
How Language Models Prioritize Contextual Grammatical Cues? Open
Transformer-based language models have shown an excellent ability to effectively capture and utilize contextual information. Although various analysis techniques have been used to quantify and trace the contribution of single contextual cu…
View article: Disentangling Textual and Acoustic Features of Neural Speech Representations
Disentangling Textual and Acoustic Features of Neural Speech Representations Open
Neural speech models build deeply entangled internal representations, which capture a variety of features (e.g., fundamental frequency, loudness, syntactic category, or semantic content of a word) in a distributed encoding. This complexity…
View article: Perception of Phonological Assimilation by Neural Speech Recognition Models
Perception of Phonological Assimilation by Neural Speech Recognition Models Open
Human listeners effortlessly compensate for phonological changes during speech perception, often unconsciously inferring the intended sounds. For example, listeners infer the underlying /n/ when hearing an utterance such as "clea[m] pan", …
View article: Encoding of lexical tone in self-supervised models of spoken language
Encoding of lexical tone in self-supervised models of spoken language Open
Interpretability research has shown that self-supervised Spoken Language Models (SLMs) encode a wide variety of features in human speech from the acoustic, phonetic, phonological, syntactic and semantic levels, to speaker characteristics. …
View article: Encoding of lexical tone in self-supervised models of spoken language
Encoding of lexical tone in self-supervised models of spoken language Open
Interpretability research has shown that self-supervised Spoken Language Models (SLMs) encode a wide variety of features in human speech from the acoustic, phonetic, phonological, syntactic and semantic levels, to speaker characteristics. …
View article: Perception of Phonological Assimilation by Neural Speech Recognition Models
Perception of Phonological Assimilation by Neural Speech Recognition Models Open
Human listeners effortlessly compensate for phonological changes during speech perception, often unconsciously inferring the intended sounds. For example, listeners infer the underlying /n/ when hearing an utterance such as “clea[m] pan”, …
View article: How Language Models Prioritize Contextual Grammatical Cues?
How Language Models Prioritize Contextual Grammatical Cues? Open
Transformer-based language models have shown an excellent ability to effectively capture and utilize contextual information. Although various analysis techniques have been used to quantify and trace the contribution of single contextual cu…
View article: Homophone Disambiguation Reveals Patterns of Context Mixing in Speech Transformers
Homophone Disambiguation Reveals Patterns of Context Mixing in Speech Transformers Open
Transformers have become a key architecture in speech processing, but our understanding of how they build up representations of acoustic and linguistic structure is limited. In this study, we address this gap by investigating how measures …
View article: Wave to Syntax: Probing spoken language models for syntax
Wave to Syntax: Probing spoken language models for syntax Open
Understanding which information is encoded in deep models of spoken and written language has been the focus of much research in recent years, as it is crucial for debugging and improving these architectures. Most previous work has focused …
View article: Wave to Syntax: Probing spoken language models for syntax
Wave to Syntax: Probing spoken language models for syntax Open
Understanding which information is encoded in deep models of spoken and written language has been the focus of much research in recent years, as it is crucial for debugging and improving these architectures. Most previous work has focused …
View article: Quantifying Context Mixing in Transformers
Quantifying Context Mixing in Transformers Open
Self-attention weights and their transformed variants have been the main source of information for analyzing token-to-token interactions in Transformer-based models. But despite their ease of interpretation, these weights are not faithful …
View article: Homophone Disambiguation Reveals Patterns of Context Mixing in Speech Transformers
Homophone Disambiguation Reveals Patterns of Context Mixing in Speech Transformers Open
Transformers have become a key architecture in speech processing, but our understanding of how they build up representations of acoustic and linguistic structure is limited. In this study, we address this gap by investigating how measures …
View article: Linguistic Productivity: the Case of Determiners in English
Linguistic Productivity: the Case of Determiners in English Open
Raquel G. Alhama, Ruthe Foushee, Daniel Byrne, Allyson Ettinger, Susan Goldin-Meadow, Afra Alishahi. Proceedings of the 13th International Joint Conference on Natural Language Processing and the 3rd Conference of the Asia-Pacific Chapter o…
View article: Quantifying Context Mixing in Transformers
Quantifying Context Mixing in Transformers Open
Self-attention weights and their transformed variants have been the main source of information for analyzing token-to-token interactions in Transformer-based models. But despite their ease of interpretation, these weights are not faithful …
View article: Learning English with Peppa Pig
Learning English with Peppa Pig Open
Recent computational models of the acquisition of spoken language via grounding in perception exploit associations between spoken and visual modalities and learn to represent speech and visual data in a joint vector space. A major unresolv…
View article: ZR-2021VG: Zero-Resource Speech Challenge, Visually-Grounded Language Modelling track, 2021 edition
ZR-2021VG: Zero-Resource Speech Challenge, Visually-Grounded Language Modelling track, 2021 edition Open
We present the visually-grounded language modelling track that was introduced in the Zero-Resource Speech challenge, 2021 edition, 2nd round. We motivate the new track and discuss participation rules in detail. We also present the two base…
View article: Discrete representations in neural models of spoken language
Discrete representations in neural models of spoken language Open
The distributed and continuous representations used by neural networks are at odds with representations employed in linguistics, which are typically symbolic. Vector quantization has been proposed as a way to induce discrete neural represe…
View article: Discrete representations in neural models of spoken language
Discrete representations in neural models of spoken language Open
The distributed and continuous representations used by neural networks are at odds with representations employed in linguistics, which are typically symbolic. Vector quantization has been proposed as a way to induce discrete neural represe…
View article: Analyzing analytical methods: The case of phonology in neural models of\n spoken language
Analyzing analytical methods: The case of phonology in neural models of\n spoken language Open
Given the fast development of analysis techniques for NLP and speech\nprocessing systems, few systematic studies have been conducted to compare the\nstrengths and weaknesses of each method. As a step in this direction we study\nthe case of…
View article: Learning to Understand Child-directed and Adult-directed Speech
Learning to Understand Child-directed and Adult-directed Speech Open
Speech directed to children differs from adult-directed speech in linguistic aspects such as repetition, word choice, and sentence length, as well as in aspects of the speech signal itself, such as prosodic and phonemic variation. Human la…
View article: Analyzing analytical methods: The case of phonology in neural models of spoken language
Analyzing analytical methods: The case of phonology in neural models of spoken language Open
Given the fast development of analysis techniques for NLP and speech processing systems, few systematic studies have been conducted to compare the strengths and weaknesses of each method. As a step in this direction we study the case of re…
View article: Bootstrapping Disjoint Datasets for Multilingual Multimodal Representation Learning
Bootstrapping Disjoint Datasets for Multilingual Multimodal Representation Learning Open
Recent work has highlighted the advantage of jointly learning grounded sentence representations from multiple languages. However, the data used in these studies has been limited to an aligned scenario: the same images annotated with senten…
View article: Curious Topics : A Curiosity-Based Model of First Language Word Learning
Curious Topics : A Curiosity-Based Model of First Language Word Learning Open
This paper investigates whether a curiosity-based strategy\ncould be beneficial to word learning. Children are active\nconversation partners and exert considerable influence over the\ntopics that are discussed in conversation with their pa…
View article: Analyzing and interpreting neural networks for NLP: A report on the first BlackboxNLP workshop
Analyzing and interpreting neural networks for NLP: A report on the first BlackboxNLP workshop Open
The Empirical Methods in Natural Language Processing (EMNLP) 2018 workshop BlackboxNLP was dedicated to resources and techniques specifically developed for analyzing and understanding the inner-workings and representations acquired by neur…
View article: Correlating neural and symbolic representations of language
Correlating neural and symbolic representations of language Open
Analysis methods which enable us to better understand the representations and functioning of neural models of language are increasingly needed as deep learning becomes the dominant approach in NLP. Here we present two methods based on Repr…
View article: Analyzing and Interpreting Neural Networks for NLP: A Report on the First BlackboxNLP Workshop
Analyzing and Interpreting Neural Networks for NLP: A Report on the First BlackboxNLP Workshop Open
The EMNLP 2018 workshop BlackboxNLP was dedicated to resources and techniques specifically developed for analyzing and understanding the inner-workings and representations acquired by neural models of language. Approaches included: systema…