Vyas Raina
YOU?
Author Swipe
View article: Embedding the Teacher: Distilling vLLM Preferences for Scalable Image Retrieval
Embedding the Teacher: Distilling vLLM Preferences for Scalable Image Retrieval Open
Text--image retrieval is necessary for applications such as product recommendation. Embedding-based approaches like CLIP enable efficient large-scale retrieval via vector similarity search, but they are primarily trained on literal caption…
View article: Universal Acoustic Adversarial Attacks for Flexible Control of Speech-LLMs
Universal Acoustic Adversarial Attacks for Flexible Control of Speech-LLMs Open
The combination of pre-trained speech encoders with large language models has enabled the development of speech LLMs that can handle a wide range of spoken language processing tasks. While these models are powerful and flexible, this very …
View article: ZeroBench: An Impossible Visual Benchmark for Contemporary Large Multimodal Models
ZeroBench: An Impossible Visual Benchmark for Contemporary Large Multimodal Models Open
Large Multimodal Models (LMMs) exhibit major shortfalls when interpreting images and, by some measures, have poorer spatial cognition than small children or animals. Despite this, they attain high scores on many popular visual benchmarks, …
View article: Universal Acoustic Adversarial Attacks for Flexible Control of Speech-LLMs
Universal Acoustic Adversarial Attacks for Flexible Control of Speech-LLMs Open
The combination of pre-trained speech encoders with large language models has enabled the development of speech LLMs that can handle a wide range of spoken language processing tasks. While these models are powerful and flexible, this very …
View article: Controlling Whisper: Universal Acoustic Adversarial Attacks to Control Speech Foundation Models
Controlling Whisper: Universal Acoustic Adversarial Attacks to Control Speech Foundation Models Open
Speech enabled foundation models, either in the form of flexible speech recognition based systems or audio-prompted large language models (LLMs), are becoming increasingly popular. One of the interesting aspects of these models is their ab…
View article: Muting Whisper: A Universal Acoustic Adversarial Attack on Speech Foundation Models
Muting Whisper: A Universal Acoustic Adversarial Attack on Speech Foundation Models Open
Recent developments in large speech foundation models like Whisper have led to their widespread use in many automatic speech recognition (ASR) applications. These systems incorporate `special tokens' in their vocabulary, such as $\texttt{<…
View article: LLM Task Interference: An Initial Study on the Impact of Task-Switch in Conversational History
LLM Task Interference: An Initial Study on the Impact of Task-Switch in Conversational History Open
With the recent emergence of powerful instruction-tuned large language models (LLMs), various helpful conversational Artificial Intelligence (AI) systems have been deployed across many applications. When prompted by users, these AI systems…
View article: Extreme Miscalibration and the Illusion of Adversarial Robustness
Extreme Miscalibration and the Illusion of Adversarial Robustness Open
Deep learning-based Natural Language Processing (NLP) models are vulnerable to adversarial attacks, where small perturbations can cause a model to misclassify. Adversarial Training (AT) is often used to increase model robustness. However, …
View article: Is LLM-as-a-Judge Robust? Investigating Universal Adversarial Attacks on Zero-shot LLM Assessment
Is LLM-as-a-Judge Robust? Investigating Universal Adversarial Attacks on Zero-shot LLM Assessment Open
Large Language Models (LLMs) are powerful zero-shot assessors used in real-world situations such as assessing written exams and benchmarking systems. Despite these critical applications, no existing work has analyzed the vulnerability of j…
View article: Blending Is All You Need: Cheaper, Better Alternative to Trillion-Parameters LLM
Blending Is All You Need: Cheaper, Better Alternative to Trillion-Parameters LLM Open
In conversational AI research, there's a noticeable trend towards developing models with a larger number of parameters, exemplified by models like ChatGPT. While these expansive models tend to generate increasingly better chat responses, t…
View article: Minimum Bayes' Risk Decoding for System Combination of Grammatical Error Correction Systems
Minimum Bayes' Risk Decoding for System Combination of Grammatical Error Correction Systems Open
For sequence-to-sequence tasks it is challenging to combine individual system outputs. Further, there is also often a mismatch between the decoding criterion and the one used for assessment. Minimum Bayes' Risk (MBR) decoding can be used t…
View article: Sample Attackability in Natural Language Adversarial Attacks
Sample Attackability in Natural Language Adversarial Attacks Open
Adversarial attack research in natural language processing (NLP) has made significant progress in designing powerful attack methods and defence approaches. However, few efforts have sought to identify which source samples are the most atta…
View article: CUED at ProbSum 2023: Hierarchical Ensemble of Summarization Models
CUED at ProbSum 2023: Hierarchical Ensemble of Summarization Models Open
In this paper, we consider the challenge of summarizing patients' medical progress notes in a limited data setting. For the Problem List Summarization (shared task 1A) at the BioNLP Workshop 2023, we demonstrate that Clinical-T5 fine-tuned…
View article: Sentiment Perception Adversarial Attacks on Neural Machine Translation Systems
Sentiment Perception Adversarial Attacks on Neural Machine Translation Systems Open
With the advent of deep learning methods, Neural Machine Translation (NMT) systems have become increasingly powerful. However, deep learning based systems are susceptible to adversarial attacks, where imperceptible changes to the input can…
View article: Rewarding Chatbots for Real-World Engagement with Millions of Users
Rewarding Chatbots for Real-World Engagement with Millions of Users Open
The emergence of pretrained large language models has led to the deployment of a range of social chatbots for chitchat. Although these chatbots demonstrate language ability and fluency, they are not guaranteed to be engaging and can strugg…
View article: Identifying Adversarially Attackable and Robust Samples
Identifying Adversarially Attackable and Robust Samples Open
Adversarial attacks insert small, imperceptible perturbations to input samples that cause large, undesired changes to the output of deep learning models. Despite extensive research on generating adversarial attacks and building defense sys…
View article: Sample Attackability in Natural Language Adversarial Attacks
Sample Attackability in Natural Language Adversarial Attacks Open
Adversarial attack research in natural language processing (NLP) has made significant progress in designing powerful attack methods and defence approaches. However, few efforts have sought to identify which source samples are the most atta…
View article: CUED at ProbSum 2023: Hierarchical Ensemble of Summarization Models
CUED at ProbSum 2023: Hierarchical Ensemble of Summarization Models Open
In this paper, we consider the challenge of summarizing patients medical progress notes in a limited data setting. For the Problem List Summarization (shared task 1A) at the BioNLP Workshop 2023, we demonstrate that ClinicalT5 fine-tuned t…
View article: Minimum Bayes’ Risk Decoding for System Combination of Grammatical Error Correction Systems
Minimum Bayes’ Risk Decoding for System Combination of Grammatical Error Correction Systems Open
For sequence-to-sequence tasks it is challenging to combine individual system outputs. Further, there is also often a mismatch between the decoding criterion and the one used for assessment. Minimum Bayes’ Risk (MBR) decoding can be used t…
View article: L2 proficiency assessment using self-supervised speech representations
L2 proficiency assessment using self-supervised speech representations Open
There has been a growing demand for automated spoken language assessment systems in recent years. A standard pipeline for this process is to start with a speech recognition system and derive features, either hand-crafted or based on deep-l…
View article: Gender Bias and Universal Substitution Adversarial Attacks on Grammatical Error Correction Systems for Automated Assessment
Gender Bias and Universal Substitution Adversarial Attacks on Grammatical Error Correction Systems for Automated Assessment Open
Grammatical Error Correction (GEC) systems perform a sequence-to-sequence task, where an input word sequence containing grammatical errors, is corrected for these errors by the GEC system to output a grammatically correct word sequence. Wi…
View article: Grammatical Error Correction Systems for Automated Assessment: Are They Susceptible to Universal Adversarial Attacks?
Grammatical Error Correction Systems for Automated Assessment: Are They Susceptible to Universal Adversarial Attacks? Open
Grammatical error correction (GEC) systems are a useful tool for assessing a learner's writing ability. These systems allow the grammatical proficiency of a candidate’s text to be assessed without requiring an examiner or teacher to read t…
View article: Residue-Based Natural Language Adversarial Attack Detection
Residue-Based Natural Language Adversarial Attack Detection Open
Deep learning based systems are susceptible to adversarial attacks, where a small, imperceptible change at the input alters the model prediction. However, to date the majority of the approaches to detect these attacks have been designed fo…
View article: Analyzing Biases to Spurious Correlations in Text Classification Tasks
Analyzing Biases to Spurious Correlations in Text Classification Tasks Open
Machine learning systems have shown impressive performance across a range of natural language tasks. However, it has been hypothesized that these systems are prone to learning spurious correlations that may be present in the training data.…
View article: Shifts: A Dataset of Real Distributional Shift Across Multiple Large-Scale Tasks
Shifts: A Dataset of Real Distributional Shift Across Multiple Large-Scale Tasks Open
There has been significant research done on developing methods for improving robustness to distributional shift and uncertainty estimation. In contrast, only limited work has examined developing standard datasets and benchmarks for assessi…
View article: Universal Adversarial Attacks on Spoken Language Assessment Systems
Universal Adversarial Attacks on Spoken Language Assessment Systems Open
There is an increasing demand for automated spoken language assessment (SLA) systems, partly driven by the performance improvements that have come from deep learning based approaches. One aspect of deep learning systems is that they do not…