Alan Ritter
YOU?
Author Swipe
View article: Camellia: Benchmarking Cultural Biases in LLMs for Asian Languages
Camellia: Benchmarking Cultural Biases in LLMs for Asian Languages Open
As Large Language Models (LLMs) gain stronger multilingual capabilities, their ability to handle culturally diverse entities becomes crucial. Prior work has shown that LLMs often favor Western-associated entities in Arabic, raising concern…
View article: SimulatorArena: Are User Simulators Reliable Proxies for Multi-Turn Evaluation of AI Assistants?
SimulatorArena: Are User Simulators Reliable Proxies for Multi-Turn Evaluation of AI Assistants? Open
Large language models (LLMs) are increasingly used in interactive applications, and human evaluation remains the gold standard for assessing their performance in multi-turn conversations. Since human studies are costly, time-consuming, and…
View article: Tree-based Dialogue Reinforced Policy Optimization for Red-Teaming Attacks
Tree-based Dialogue Reinforced Policy Optimization for Red-Teaming Attacks Open
Despite recent rapid progress in AI safety, current large language models remain vulnerable to adversarial attacks in multi-turn interaction settings, where attackers strategically adapt their prompts across conversation turns and pose a m…
View article: MS-CLR: Multi-Skeleton Contrastive Learning for Human Action Recognition
MS-CLR: Multi-Skeleton Contrastive Learning for Human Action Recognition Open
Contrastive learning has gained significant attention in skeleton-based action recognition for its ability to learn robust representations from unlabeled data. However, existing methods rely on a single skeleton convention, which limits th…
View article: Tabular Data Understanding with LLMs: A Survey of Recent Advances and Challenges
Tabular Data Understanding with LLMs: A Survey of Recent Advances and Challenges Open
Tables have gained significant attention in large language models (LLMs) and multimodal large language models (MLLMs) due to their complex and flexible structure. Unlike linear text inputs, tables are two-dimensional, encompassing formats …
View article: Measuring, Modeling, and Helping People Account for Privacy Risks in Online Self-Disclosures with AI
Measuring, Modeling, and Helping People Account for Privacy Risks in Online Self-Disclosures with AI Open
In pseudonymous online fora like Reddit, the benefits of self-disclosure are often apparent to users (e.g., I can vent about my in-laws to understanding strangers), but the privacy risks are more abstract (e.g., will my partner be able to …
View article: CROSSNEWS: A Cross-Genre Authorship Verification and Attribution Benchmark
CROSSNEWS: A Cross-Genre Authorship Verification and Attribution Benchmark Open
Authorship models have historically generalized poorly to new domains because of the wide distribution of author-identifying signals across domains. In particular, the effects of topic and genre are highly domain-dependent and impact autho…
View article: How to Protect Yourself from 5G Radiation? Investigating LLM Responses to Implicit Misinformation
How to Protect Yourself from 5G Radiation? Investigating LLM Responses to Implicit Misinformation Open
As Large Language Models (LLMs) are widely deployed in diverse scenarios, the extent to which they could tacitly spread misinformation emerges as a critical safety concern. Current research primarily evaluates LLMs on explicit false statem…
View article: Probabilistic Reasoning with LLMs for k-anonymity Estimation
Probabilistic Reasoning with LLMs for k-anonymity Estimation Open
Probabilistic reasoning is a key aspect of both human and artificial intelligence that allows for handling uncertainty and ambiguity in decision-making. In this paper, we introduce a new numerical reasoning task under uncertainty for large…
View article: Language Models can Self-Improve at State-Value Estimation for Better Search
Language Models can Self-Improve at State-Value Estimation for Better Search Open
Collecting ground-truth rewards or human demonstrations for multi-step reasoning tasks is often prohibitively expensive, particularly in interactive domains such as web tasks. We introduce Self-Taught Lookahead (STL), a reward-free framewo…
View article: Can LLMs Help Uncover Insights about LLMs? A Large-Scale, Evolving Literature Analysis of Frontier LLMs
Can LLMs Help Uncover Insights about LLMs? A Large-Scale, Evolving Literature Analysis of Frontier LLMs Open
The surge of LLM studies makes synthesizing their findings challenging. Analysis of experimental results from literature can uncover important trends across studies, but the time-consuming nature of manual data extraction limits its use. O…
View article: What are Foundation Models Cooking in the Post-Soviet World?
What are Foundation Models Cooking in the Post-Soviet World? Open
The culture of the Post-Soviet states is complex, shaped by a turbulent history that continues to influence current events. In this study, we investigate the Post-Soviet cultural food knowledge of foundation models by constructing BORSch, …
View article: Measuring, Modeling, and Helping People Account for Privacy Risks in Online Self-Disclosures with AI
Measuring, Modeling, and Helping People Account for Privacy Risks in Online Self-Disclosures with AI Open
In pseudonymous online fora like Reddit, the benefits of self-disclosure are often apparent to users (e.g., I can vent about my in-laws to understanding strangers), but the privacy risks are more abstract (e.g., will my partner be able to …
View article: Granular Privacy Control for Geolocation with Vision Language Models
Granular Privacy Control for Geolocation with Vision Language Models Open
Vision Language Models (VLMs) are rapidly advancing in their capability to answer information-seeking questions. As these models are widely deployed in consumer applications, they could lead to new privacy risks due to emergent abilities t…
View article: Self-MoE: Towards Compositional Large Language Models with Self-Specialized Experts
Self-MoE: Towards Compositional Large Language Models with Self-Specialized Experts Open
We present Self-MoE, an approach that transforms a monolithic LLM into a compositional, modular system of self-specialized experts, named MiXSE (MiXture of Self-specialized Experts). Our approach leverages self-specialization, which constr…
View article: NEO-BENCH: Evaluating Robustness of Large Language Models with Neologisms
NEO-BENCH: Evaluating Robustness of Large Language Models with Neologisms Open
The performance of Large Language Models (LLMs) degrades from the temporal drift between data used for model training and newer text seen during inference. One understudied avenue of language change causing data drift is the emergence of n…
View article: Stanceosaurus 2.0: Classifying Stance Towards Russian and Spanish Misinformation
Stanceosaurus 2.0: Classifying Stance Towards Russian and Spanish Misinformation Open
The Stanceosaurus corpus (Zheng et al., 2022) was designed to provide high-quality, annotated, 5-way stance data extracted from Twitter, suitable for analyzing cross-cultural and cross-lingual misinformation. In the Stanceosaurus 2.0 itera…
View article: Constrained Decoding for Cross-lingual Label Projection
Constrained Decoding for Cross-lingual Label Projection Open
Zero-shot cross-lingual transfer utilizing multilingual LLMs has become a popular learning paradigm for low-resource languages with no labeled training data. However, for NLP tasks that involve fine-grained predictions on words and phrases…
View article: UniIR: Training and Benchmarking Universal Multimodal Information Retrievers
UniIR: Training and Benchmarking Universal Multimodal Information Retrievers Open
Existing information retrieval (IR) models often assume a homogeneous format, limiting their applicability to diverse user needs, such as searching for images with text descriptions, searching for a news article with a headline image, or f…
View article: Reducing Privacy Risks in Online Self-Disclosures with Language Models
Reducing Privacy Risks in Online Self-Disclosures with Language Models Open
Self-disclosure, while being common and rewarding in social media interaction, also poses privacy risks. In this paper, we take the initiative to protect the user-side privacy associated with online self-disclosure through detection and ab…
View article: Tensor Trust: Interpretable Prompt Injection Attacks from an Online Game
Tensor Trust: Interpretable Prompt Injection Attacks from an Online Game Open
While Large Language Models (LLMs) are increasingly being used in real-world applications, they remain vulnerable to prompt injection attacks: malicious third party prompts that subvert the intent of the system designer. To help researcher…