Vivek Srikumar
YOU?
Author Swipe
View article: Reinforcing Code Generation: Improving Text-to-SQL with Execution-Based Learning
Reinforcing Code Generation: Improving Text-to-SQL with Execution-Based Learning Open
In this work, we study the problem of code generation with a large language model (LLM), with a focus on generating SQL queries from natural language questions. We ask: Instead of using supervised fine tuning with text-code pairs, can we t…
View article: LLM-Symbolic Integration for Robust Temporal Tabular Reasoning
LLM-Symbolic Integration for Robust Temporal Tabular Reasoning Open
Temporal tabular question answering presents a significant challenge for Large Language Models (LLMs), requiring robust reasoning over structured data, which is a task where traditional prompting methods often fall short. These methods fac…
View article: Found in Translation: Measuring Multilingual LLM Consistency as Simple as Translate then Evaluate
Found in Translation: Measuring Multilingual LLM Consistency as Simple as Translate then Evaluate Open
Large language models (LLMs) provide detailed and impressive responses to queries in English. However, are they really consistent at responding to the same query in other languages? The popular way of evaluating for multilingual performanc…
View article: Test-Time Scaling with Repeated Sampling Improves Multilingual Text Generation
Test-Time Scaling with Repeated Sampling Improves Multilingual Text Generation Open
Inference-time scaling via repeated sampling has shown promise in reasoning tasks, but its effectiveness in multilingual generation remains underexplored. We evaluate this approach using perplexity- and reward-based verifiers on two multil…
View article: A Survey of Model Architectures in Information Retrieval
A Survey of Model Architectures in Information Retrieval Open
The period from 2019 to the present has represented one of the biggest paradigm shifts in information retrieval (IR) and natural language processing (NLP), culminating in the emergence of powerful large language models (LLMs) from 2022 onw…
View article: Understanding the Logic of Direct Preference Alignment through Logic
Understanding the Logic of Direct Preference Alignment through Logic Open
Recent direct preference alignment algorithms (DPA), such as DPO, have shown great promise in aligning large language models to human preferences. While this has motivated the development of many new variants of the original DPO loss, unde…
View article: State Space Models are Strong Text Rerankers
State Space Models are Strong Text Rerankers Open
Transformers dominate NLP and IR; but their inference inefficiencies and challenges in extrapolating to longer contexts have sparked interest in alternative model architectures. Among these, state space models (SSMs) like Mamba offer promi…
View article: CrysText: A Generative AI Approach for Text-Conditioned Crystal Structure Generation using LLM
CrysText: A Generative AI Approach for Text-Conditioned Crystal Structure Generation using LLM Open
Generating crystal structures directly from textual descriptions marks a pivotal advancement in materials informatics, offering a streamlined pathway from concept to discovery. Integrating generative models into Crystal Structure Predictio…
View article: Identification of cultural conversations in therapy using natural language processing models.
Identification of cultural conversations in therapy using natural language processing models. Open
Researchers have historically focused on understanding therapist multicultural competency and orientation through client self-report measures and behavioral coding. While client perceptions of therapist cultural competency and multicultura…
View article: Beyond Perplexity: Multi-dimensional Safety Evaluation of LLM Compression
Beyond Perplexity: Multi-dimensional Safety Evaluation of LLM Compression Open
Increasingly, model compression techniques enable large language models (LLMs) to be deployed in real-world applications. As a result of this momentum towards local deployment, compressed LLMs will interact with a large population. Prior w…
View article: An Empirical Investigation of Matrix Factorization Methods for Pre-trained Transformers
An Empirical Investigation of Matrix Factorization Methods for Pre-trained Transformers Open
The increasing size of transformer-based models in NLP makes the question of compressing them important. In this work, we present a comprehensive analysis of factorization based model compression techniques. Specifically, we focus on compa…
View article: An In-depth Investigation of User Response Simulation for Conversational Search
An In-depth Investigation of User Response Simulation for Conversational Search Open
Conversational search has seen increased recent attention in both the IR and NLP communities. It seeks to clarify and solve users' search needs through multi-turn natural language interactions. However, most existing systems are trained an…
View article: Calibrating Large Language Models Using Their Generations Only
Calibrating Large Language Models Using Their Generations Only Open
As large language models (LLMs) are increasingly deployed in user-facing applications, building trust and maintaining safety by accurately quantifying a model's confidence in its prediction becomes even more important. However, finding eff…
View article: TRAP: Targeted Random Adversarial Prompt Honeypot for Black-Box Identification
TRAP: Targeted Random Adversarial Prompt Honeypot for Black-Box Identification Open
Large Language Model (LLM) services and models often come with legal rules on who can use them and how they must use them. Assessing the compliance of the released LLMs is crucial, as these rules protect the interests of the LLM contributo…
View article: In-Context Example Ordering Guided by Label Distributions
In-Context Example Ordering Guided by Label Distributions Open
By allowing models to predict without task-specific training, in-context learning (ICL) with pretrained LLMs has enormous potential in NLP. However, a number of problems persist in ICL. In particular, its performance is sensitive to the ch…
View article: Promptly Predicting Structures: The Return of Inference
Promptly Predicting Structures: The Return of Inference Open
Prompt-based methods have been used extensively across NLP to build zero- and few-shot label predictors. Many NLP tasks are naturally structured: that is, their outputs consist of multiple labels which constrain each other. Annotating data…
View article: MultiLegalPile: A 689GB Multilingual Legal Corpus
MultiLegalPile: A 689GB Multilingual Legal Corpus Open
Large, high-quality datasets are crucial for training Large Language Models (LLMs). However, so far, there are few datasets available for specialized critical domains such as law and the available ones are often only for the English langua…