Ximing Lu
YOU?
Author Swipe
View article: DLER: Doing Length pEnalty Right - Incentivizing More Intelligence per Token via Reinforcement Learning
DLER: Doing Length pEnalty Right - Incentivizing More Intelligence per Token via Reinforcement Learning Open
Reasoning language models such as OpenAI-o1, DeepSeek-R1, and Qwen achieve strong performance via extended chains of thought but often generate unnecessarily long outputs. Maximizing intelligence per token--accuracy relative to response le…
View article: BroRL: Scaling Reinforcement Learning via Broadened Exploration
BroRL: Scaling Reinforcement Learning via Broadened Exploration Open
Reinforcement Learning with Verifiable Rewards (RLVR) has emerged as a key ingredient for unlocking complex reasoning capabilities in large language models. Recent work ProRL has shown promise in scaling RL by increasing the number of trai…
View article: DeepSearch: Overcome the Bottleneck of Reinforcement Learning with Verifiable Rewards via Monte Carlo Tree Search
DeepSearch: Overcome the Bottleneck of Reinforcement Learning with Verifiable Rewards via Monte Carlo Tree Search Open
Although RLVR has become an essential component for developing advanced reasoning skills in LLMs, contemporary studies have documented training plateaus that emerge following thousands of optimization steps, demonstrating notable decreases…
View article: Bias in Gender Bias Benchmarks: How Spurious Features Distort Evaluation
Bias in Gender Bias Benchmarks: How Spurious Features Distort Evaluation Open
Gender bias in vision-language foundation models (VLMs) raises concerns about their safe deployment and is typically evaluated using benchmarks with gender annotations on real-world images. However, as these benchmarks often contain spurio…
View article: Scaling Up RL: Unlocking Diverse Reasoning in LLMs via Prolonged Training
Scaling Up RL: Unlocking Diverse Reasoning in LLMs via Prolonged Training Open
Recent advancements in reasoning-focused language models such as OpenAI's O1 and DeepSeek-R1 have shown that scaling test-time computation-through chain-of-thought reasoning and iterative exploration-can yield substantial improvements on c…
View article: Verifying the Verifiers: Unveiling Pitfalls and Potentials in Fact Verifiers
Verifying the Verifiers: Unveiling Pitfalls and Potentials in Fact Verifiers Open
Fact verification is essential for ensuring the reliability of LLM applications. In this study, we evaluate 12 pre-trained LLMs and one specialized fact-verifier, including frontier LLMs and open-weight reasoning LLMs, using a collection o…
View article: ProRL: Prolonged Reinforcement Learning Expands Reasoning Boundaries in Large Language Models
ProRL: Prolonged Reinforcement Learning Expands Reasoning Boundaries in Large Language Models Open
Recent advances in reasoning-centric language models have highlighted reinforcement learning (RL) as a promising method for aligning models with verifiable rewards. However, it remains contentious whether RL truly expands a model's reasoni…
View article: Prismatic Synthesis: Gradient-based Data Diversification Boosts Generalization in LLM Reasoning
Prismatic Synthesis: Gradient-based Data Diversification Boosts Generalization in LLM Reasoning Open
Effective generalization in language models depends critically on the diversity of their training data. Yet existing diversity metrics often fall short of this goal, relying on surface-level heuristics that are decoupled from model behavio…
View article: Information-Guided Identification of Training Data Imprint in (Proprietary) Large Language Models
Information-Guided Identification of Training Data Imprint in (Proprietary) Large Language Models Open
View article: Socratic-MCTS: Test-Time Visual Reasoning by Asking the Right Questions
Socratic-MCTS: Test-Time Visual Reasoning by Asking the Right Questions Open
View article: AI as Humanity's Salieri: Quantifying Linguistic Creativity of Language Models via Systematic Attribution of Machine Text against Web Text
AI as Humanity's Salieri: Quantifying Linguistic Creativity of Language Models via Systematic Attribution of Machine Text against Web Text Open
Creativity has long been considered one of the most difficult aspect of human intelligence for AI to mimic. However, the rise of Large Language Models (LLMs), like ChatGPT, has raised questions about whether AI can match or even surpass hu…
View article: HAICOSYSTEM: An Ecosystem for Sandboxing Safety Risks in Human-AI Interactions
HAICOSYSTEM: An Ecosystem for Sandboxing Safety Risks in Human-AI Interactions Open
AI agents are increasingly autonomous in their interactions with human users and tools, leading to increased interactional safety risks. We present HAICOSYSTEM, a framework examining AI agent safety within diverse and complex social intera…
View article: StyleRemix: Interpretable Authorship Obfuscation via Distillation and Perturbation of Style Elements
StyleRemix: Interpretable Authorship Obfuscation via Distillation and Perturbation of Style Elements Open
Authorship obfuscation, rewriting a text to intentionally obscure the identity of the author, is an important but challenging task. Current methods using large language models (LLMs) lack interpretability and controllability, often ignorin…
View article: Certainly Uncertain: A Benchmark and Metric for Multimodal Epistemic and Aleatoric Awareness
Certainly Uncertain: A Benchmark and Metric for Multimodal Epistemic and Aleatoric Awareness Open
The ability to acknowledge the inevitable uncertainty in their knowledge and reasoning is a prerequisite for AI systems to be truly truthful and reliable. In this paper, we present a taxonomy of uncertainty specific to vision-language AI s…
View article: How to Train Your Fact Verifier: Knowledge Transfer with Multimodal Open Models
How to Train Your Fact Verifier: Knowledge Transfer with Multimodal Open Models Open
Given the growing influx of misinformation across news and social media, there is a critical need for systems that can provide effective real-time verification of news claims. Large language or multimodal model based verification has been …
View article: WildTeaming at Scale: From In-the-Wild Jailbreaks to (Adversarially) Safer Language Models
WildTeaming at Scale: From In-the-Wild Jailbreaks to (Adversarially) Safer Language Models Open
We introduce WildTeaming, an automatic LLM safety red-teaming framework that mines in-the-wild user-chatbot interactions to discover 5.7K unique clusters of novel jailbreak tactics, and then composes multiple tactics for systematic explora…
View article: Value Kaleidoscope: Engaging AI with Pluralistic Human Values, Rights, and Duties
Value Kaleidoscope: Engaging AI with Pluralistic Human Values, Rights, and Duties Open
Human values are crucial to human decision-making. Value pluralism is the view that multiple correct values may be held in tension with one another (e.g., when considering lying to a friend to protect their feelings, how does one balance h…
View article: Information-Theoretic Distillation for Reference-less Summarization
Information-Theoretic Distillation for Reference-less Summarization Open
The current winning recipe for automatic summarization is using proprietary large-scale language models (LLMs) such as ChatGPT as is, or imitation learning from them as teacher models. While increasingly ubiquitous dependence on such large…
View article: Logic-Induced-Long-Tail (LINT)
Logic-Induced-Long-Tail (LINT) Open
Data release for Arxiv Paper: In Search of the Long-Tail: Systematic Generation of Long-Tail Inferential Knowledge via Logical Rule Guided Search
View article: JAMDEC: Unsupervised Authorship Obfuscation using Constrained Decoding over Small Language Models
JAMDEC: Unsupervised Authorship Obfuscation using Constrained Decoding over Small Language Models Open
The permanence of online content combined with the enhanced authorship identification techniques calls for stronger computational methods to protect the identity and privacy of online authorship when needed, e.g., blind reviews for scienti…
View article: NovaCOMET: Open Commonsense Foundation Models with Symbolic Knowledge Distillation
NovaCOMET: Open Commonsense Foundation Models with Symbolic Knowledge Distillation Open
We present NovaCOMET, an open commonsense knowledge model, that combines the best aspects of knowledge and general task models. Compared to previous knowledge models, NovaCOMET allows open-format relations enabling direct application to re…
View article: Localized Symbolic Knowledge Distillation for Visual Commonsense Models
Localized Symbolic Knowledge Distillation for Visual Commonsense Models Open
Instruction following vision-language (VL) models offer a flexible interface that supports a broad range of multimodal tasks in a zero-shot fashion. However, interfaces that operate on full images do not directly enable the user to "point …
View article: The Unlocking Spell on Base LLMs: Rethinking Alignment via In-Context Learning
The Unlocking Spell on Base LLMs: Rethinking Alignment via In-Context Learning Open
The alignment tuning process of large language models (LLMs) typically involves instruction learning through supervised fine-tuning (SFT) and preference tuning via reinforcement learning from human feedback (RLHF). A recent study, LIMA (Zh…
View article: STEER: Unified Style Transfer with Expert Reinforcement
STEER: Unified Style Transfer with Expert Reinforcement Open
While text style transfer has many applications across natural language processing, the core premise of transferring from a single source style is unrealistic in a real-world setting. In this work, we focus on arbitrary style transfer: rew…
View article: Tailoring Self-Rationalizers with Multi-Reward Distillation
Tailoring Self-Rationalizers with Multi-Reward Distillation Open
Large language models (LMs) are capable of generating free-text rationales to aid question answering. However, prior work 1) suggests that useful self-rationalization is emergent only at significant scales (e.g., 175B parameter GPT-3); and…
View article: The Generative AI Paradox: "What It Can Create, It May Not Understand"
The Generative AI Paradox: "What It Can Create, It May Not Understand" Open
The recent wave of generative AI has sparked unprecedented global attention, with both excitement and concern over potentially superhuman levels of artificial intelligence: models now take only seconds to produce outputs that would challen…
View article: Phenomenal Yet Puzzling: Testing Inductive Reasoning Capabilities of Language Models with Hypothesis Refinement
Phenomenal Yet Puzzling: Testing Inductive Reasoning Capabilities of Language Models with Hypothesis Refinement Open
The ability to derive underlying principles from a handful of observations and then generalize to novel situations -- known as inductive reasoning -- is central to human intelligence. Prior work suggests that language models (LMs) often fa…
View article: Value Kaleidoscope: Engaging AI with Pluralistic Human Values, Rights, and Duties
Value Kaleidoscope: Engaging AI with Pluralistic Human Values, Rights, and Duties Open
Human values are crucial to human decision-making. Value pluralism is the view that multiple correct values may be held in tension with one another (e.g., when considering lying to a friend to protect their feelings, how does one balance h…
View article: Faith and Fate: Limits of Transformers on Compositionality
Faith and Fate: Limits of Transformers on Compositionality Open
Transformer large language models (LLMs) have sparked admiration for their exceptional performance on tasks that demand intricate multi-step reasoning. Yet, these models simultaneously show failures on surprisingly trivial problems. This b…
View article: Impossible Distillation: from Low-Quality Model to High-Quality Dataset & Model for Summarization and Paraphrasing
Impossible Distillation: from Low-Quality Model to High-Quality Dataset & Model for Summarization and Paraphrasing Open
We present Impossible Distillation, a novel framework for paraphrasing and sentence summarization, that distills a high-quality dataset and model from a low-quality teacher that itself cannot perform these tasks. Unlike prior works that re…