Subba Reddy Oota
YOU?
Author Swipe
View article: Brain-Informed Fine-Tuning for Improved Multilingual Understanding in Language Models
Brain-Informed Fine-Tuning for Improved Multilingual Understanding in Language Models Open
Recent studies have demonstrated that fine-tuning language models with brain data can improve their semantic understanding, although these findings have so far been limited to English. Interestingly, similar to the shared multilingual embe…
View article: Correlating instruction-tuning (in multimodal models) with vision-language processing (in the brain)
Correlating instruction-tuning (in multimodal models) with vision-language processing (in the brain) Open
Transformer-based language models, though not explicitly trained to mimic brain recordings, have demonstrated surprising alignment with brain activity. Progress in these models-through increased size, instruction-tuning, and multimodality-…
View article: Multi-modal brain encoding models for multi-modal stimuli
Multi-modal brain encoding models for multi-modal stimuli Open
Despite participants engaging in unimodal stimuli, such as watching images or silent videos, recent work has demonstrated that multi-modal Transformer models can predict visual brain activity impressively well, even with incongruent modali…
View article: Aligning Text/Speech Representations from Multimodal Models with MEG Brain Activity During Listening
Aligning Text/Speech Representations from Multimodal Models with MEG Brain Activity During Listening Open
View article: USDC: A Dataset of User Stance and Dogmatism in Long Conversations
USDC: A Dataset of User Stance and Dogmatism in Long Conversations Open
View article: Language Models and Brain Alignment: Brain Encoding and Decoding
Language Models and Brain Alignment: Brain Encoding and Decoding Open
View article: IndicSentEval: How Effectively do Multilingual Transformer Models encode Linguistic Properties for Indic Languages?
IndicSentEval: How Effectively do Multilingual Transformer Models encode Linguistic Properties for Indic Languages? Open
Transformer-based models have revolutionized the field of natural language processing. To understand why they perform so well and to assess their reliability, several studies have focused on questions such as: Which linguistic properties a…
View article: USDC: A Dataset of $\underline{U}$ser $\underline{S}$tance and $\underline{D}$ogmatism in Long $\underline{C}$onversations
USDC: A Dataset of $\underline{U}$ser $\underline{S}$tance and $\underline{D}$ogmatism in Long $\underline{C}$onversations Open
Analyzing user opinion changes in long conversation threads is extremely critical for applications like enhanced personalization, market research, political campaigns, customer service, targeted advertising, and content moderation. Unfortu…
View article: Modèles neurocomputationnels de la compréhension du langage : caractérisation des similarités et des différences entre le traitement cérébral du langage et les modèles de langage
Modèles neurocomputationnels de la compréhension du langage : caractérisation des similarités et des différences entre le traitement cérébral du langage et les modèles de langage Open
This thesis explores the synergy between artificial intelligence (AI) and cognitive neuroscience to advance language processing capabilities. It builds on the insight that breakthroughs in AI, such as convolutional neural networks and mech…
View article: Attention-based fusion of multiple graphheat networks for structural to functional brain mapping
Attention-based fusion of multiple graphheat networks for structural to functional brain mapping Open
Over the last decade, there has been growing interest in learning the mapping from structural connectivity (SC) to functional connectivity (FC) of the brain. The spontaneous fluctuations of the brain activity during the resting-state as ca…
View article: Modelling Cross-Situational Learning on Full Sentences in Few Shots with Simple RNNs
Modelling Cross-Situational Learning on Full Sentences in Few Shots with Simple RNNs Open
How do children bootstrap language through noisy supervision? Most prior works focused on tracking co-occurrences between individual words and referents. We model cross-situational learning (CSL) at sentence level with few (1000) training …
View article: Speech language models lack important brain-relevant semantics
Speech language models lack important brain-relevant semantics Open
Despite known differences between reading and listening in the brain, recent work has shown that text-based language models predict both text-evoked and speech-evoked brain activity to an impressive degree. This poses the question of what …
View article: MEG Encoding using Word Context Semantics in Listening Stories
MEG Encoding using Word Context Semantics in Listening Stories Open
International audience
View article: Speech Taskonomy: Which Speech Tasks are the most Predictive of fMRI Brain Activity?
Speech Taskonomy: Which Speech Tasks are the most Predictive of fMRI Brain Activity? Open
International audience
View article: Deep Neural Networks and Brain Alignment: Brain Encoding and Decoding (Survey)
Deep Neural Networks and Brain Alignment: Brain Encoding and Decoding (Survey) Open
Can artificial intelligence unlock the secrets of the human brain? How do the inner mechanisms of deep learning models relate to our neural circuits? Is it possible to enhance AI by tapping into the power of brain recordings? These captiva…
View article: On Robustness of Finetuned Transformer-based NLP Models
On Robustness of Finetuned Transformer-based NLP Models Open
Transformer-based pretrained models like BERT, GPT-2 and T5 have been finetuned for a large number of natural language processing (NLP) tasks, and have been shown to be very effective. However, while finetuning, what changes across layers …
View article: Syntactic Structure Processing in the Brain while Listening
Syntactic Structure Processing in the Brain while Listening Open
Syntactic parsing is the task of assigning a syntactic structure to a sentence. There are two popular syntactic parsing methods: constituency and dependency parsing. Recent works have used syntactic embeddings based on constituency trees, …
View article: On Robustness of Finetuned Transformer-based NLP Models
On Robustness of Finetuned Transformer-based NLP Models Open
Transformer-based pretrained models like BERT, GPT-2 and T5 have been finetuned for a large number of natural language processing (NLP) tasks, and have been shown to be very effective. However, while finetuning, what changes across layers …
View article: What aspects of NLP models and brain datasets affect brain-NLP alignment?
What aspects of NLP models and brain datasets affect brain-NLP alignment? Open
International audience
View article: How does the brain process syntactic structure while listening?
How does the brain process syntactic structure while listening? Open
International audience
View article: GAE-ISumm: Unsupervised Graph-Based Summarization of Indian Languages
GAE-ISumm: Unsupervised Graph-Based Summarization of Indian Languages Open
Document summarization aims to create a precise and coherent summary of a text document. Many deep learning summarization models are developed mainly for English, often requiring a large training corpus and efficient pre-trained language m…
View article: Joint processing of linguistic properties in brains and language models
Joint processing of linguistic properties in brains and language models Open
Language models have been shown to be very effective in predicting brain recordings of subjects experiencing complex language stimuli. For a deeper understanding of this alignment, it is important to understand the correspondence between t…
View article: How distinct are Syntactic and Semantic Representations in the Brain During Sentence Comprehension?
How distinct are Syntactic and Semantic Representations in the Brain During Sentence Comprehension? Open
View article: Long Short-Term Memory of Language Models for Predicting Brain Activation During Listening to Stories
Long Short-Term Memory of Language Models for Predicting Brain Activation During Listening to Stories Open
International audience
View article: Investigating Long-Term Context of Language Models on Brain Activity during Narrative Listening in fMRI
Investigating Long-Term Context of Language Models on Brain Activity during Narrative Listening in fMRI Open
View article: Neural Language Taskonomy: Which NLP Tasks are the most Predictive of\n fMRI Brain Activity?
Neural Language Taskonomy: Which NLP Tasks are the most Predictive of\n fMRI Brain Activity? Open
Several popular Transformer based language models have been found to be\nsuccessful for text-driven brain encoding. However, existing literature\nleverages only pretrained text Transformer models and has not explored the\nefficacy of task-…
View article: Multi-Task Text Classification using Graph Convolutional Networks for Large-Scale Low Resource Language
Multi-Task Text Classification using Graph Convolutional Networks for Large-Scale Low Resource Language Open
Graph Convolutional Networks (GCN) have achieved state-of-art results on single text classification tasks like sentiment analysis, emotion detection, etc. However, the performance is achieved by testing and reporting on resource-rich langu…
View article: Visio-Linguistic Brain Encoding
Visio-Linguistic Brain Encoding Open
Enabling effective brain-computer interfaces requires understanding how the\nhuman brain encodes stimuli across modalities such as visual, language (or\ntext), etc. Brain encoding aims at constructing fMRI brain activity given a\nstimulus.…
View article: Cross-view Brain Decoding
Cross-view Brain Decoding Open
How the brain captures the meaning of linguistic stimuli across multiple views is still a critical open question in neuroscience. Consider three different views of the concept apartment: (1) picture (WP) presented with the target word labe…
View article: Visio-Linguistic Brain Encoding
Visio-Linguistic Brain Encoding Open
Enabling effective brain-computer interfaces requires understanding how the human brain encodes stimuli across modalities such as visual, language (or text), etc. Brain encoding aims at constructing fMRI brain activity given a stimulus. Th…