Daking Rai
YOU?
Author Swipe
View article: Failure by Interference: Language Models Make Balanced Parentheses Errors When Faulty Mechanisms Overshadow Sound Ones
Failure by Interference: Language Models Make Balanced Parentheses Errors When Faulty Mechanisms Overshadow Sound Ones Open
Despite remarkable advances in coding capabilities, language models (LMs) still struggle with simple syntactic tasks such as generating balanced parentheses. In this study, we investigate the underlying mechanisms behind the persistence of…
View article: Mechanistic Understanding of Language Models in Syntactic Code Completion
Mechanistic Understanding of Language Models in Syntactic Code Completion Open
Recently, language models (LMs) have shown impressive proficiency in code generation tasks, especially when fine-tuned on code-specific datasets, commonly known as Code LMs. However, our understanding of the internal decision-making proces…
View article: All for One: LLMs Solve Mental Math at the Last Token With Information Transferred From Other Tokens
All for One: LLMs Solve Mental Math at the Last Token With Information Transferred From Other Tokens Open
View article: A Survey on Sparse Autoencoders: Interpreting the Internal Mechanisms of Large Language Models
A Survey on Sparse Autoencoders: Interpreting the Internal Mechanisms of Large Language Models Open
View article: Understanding the Effect of Algorithm Transparency of Model Explanations in Text-to-SQL Semantic Parsing
Understanding the Effect of Algorithm Transparency of Model Explanations in Text-to-SQL Semantic Parsing Open
Explaining the decisions of AI has become vital for fostering appropriate user trust in these systems. This paper investigates explanations for a structured prediction task called ``text-to-SQL Semantic Parsing'', which translates a natura…
View article: A Practical Review of Mechanistic Interpretability for Transformer-Based Language Models
A Practical Review of Mechanistic Interpretability for Transformer-Based Language Models Open
Mechanistic interpretability (MI) is an emerging sub-field of interpretability that seeks to understand a neural network model by reverse-engineering its internal computations. Recently, MI has garnered significant attention for interpreti…
View article: Explaining Large Language Model-Based Neural Semantic Parsers (Student Abstract)
Explaining Large Language Model-Based Neural Semantic Parsers (Student Abstract) Open
While large language models (LLMs) have demonstrated strong capability in structured prediction tasks such as semantic parsing, few amounts of research have explored the underlying mechanisms of their success. Our work studies different me…
View article: Improving Generalization in Language Model-Based Text-to-SQL Semantic Parsing: Two Simple Semantic Boundary-Based Techniques
Improving Generalization in Language Model-Based Text-to-SQL Semantic Parsing: Two Simple Semantic Boundary-Based Techniques Open
Compositional and domain generalization present significant challenges in semantic parsing, even for state-of-the-art semantic parsers based on pre-trained language models (LMs). In this study, we empirically investigate improving an LM's …
View article: Explaining Large Language Model-Based Neural Semantic Parsers (Student Abstract)
Explaining Large Language Model-Based Neural Semantic Parsers (Student Abstract) Open
While large language models (LLMs) have demonstrated strong capability in structured prediction tasks such as semantic parsing, few amounts of research have explored the underlying mechanisms of their success. Our work studies different me…
View article: Improving Generalization in Language Model-based Text-to-SQL Semantic Parsing: Two Simple Semantic Boundary-based Techniques
Improving Generalization in Language Model-based Text-to-SQL Semantic Parsing: Two Simple Semantic Boundary-based Techniques Open
Compositional and domain generalization present significant challenges in semantic parsing, even for state-of-the-art semantic parsers based on pre-trained language models (LMs). In this study, we empirically investigate improving an LM's …