Qingru Zhang
YOU?
Author Swipe
View article: SMFR-Net: simple multi-domain flare removal network
SMFR-Net: simple multi-domain flare removal network Open
When strong light enters the lens, multiple internal reflections and scattering can cause flare, significantly degrading image quality and affecting the performance of downstream vision tasks. In practical photography, flare is often cause…
View article: Ask a Strong LLM Judge when Your Reward Model is Uncertain
Ask a Strong LLM Judge when Your Reward Model is Uncertain Open
Reward model (RM) plays a pivotal role in reinforcement learning with human feedback (RLHF) for aligning large language models (LLMs). However, classical RMs trained on human preferences are vulnerable to reward hacking and generalize poor…
View article: Hybrid Convolutional Transformer with Dynamic Prompting for Adaptive Image Restoration
Hybrid Convolutional Transformer with Dynamic Prompting for Adaptive Image Restoration Open
High-quality image restoration (IR) is a fundamental task in computer vision, aiming to recover a clear image from its degraded version. Prevailing methods typically employ a static inference pipeline, neglecting the spatial variability of…
View article: Win Fast or Lose Slow: Balancing Speed and Accuracy in Latency-Sensitive Decisions of LLMs
Win Fast or Lose Slow: Balancing Speed and Accuracy in Latency-Sensitive Decisions of LLMs Open
Large language models (LLMs) have shown remarkable performance across diverse reasoning and generation tasks, and are increasingly deployed as agents in dynamic environments such as code generation and recommendation systems. However, many…
View article: Think-RM: Enabling Long-Horizon Reasoning in Generative Reward Models
Think-RM: Enabling Long-Horizon Reasoning in Generative Reward Models Open
Reinforcement learning from human feedback (RLHF) has become a powerful post-training paradigm for aligning large language models with human preferences. A core challenge in RLHF is constructing accurate reward signals, where the conventio…
View article: Multiscale Spatio-Temporal Fusion Network for Video Dehazing
Multiscale Spatio-Temporal Fusion Network for Video Dehazing Open
View article: Reform of Traditional Music Teaching Methods and Cultivation of Students’ Musical Creativity on Digital Platforms
Reform of Traditional Music Teaching Methods and Cultivation of Students’ Musical Creativity on Digital Platforms Open
Due to the rapid development of modern science and technology, the teaching methods and modes in traditional music have become increasingly obsolete and outdated, and digital teaching resources are the key to the reform of traditional musi…
View article: DORM: Preference Data Weights Optimization for Reward Modeling in LLM Alignment
DORM: Preference Data Weights Optimization for Reward Modeling in LLM Alignment Open
View article: Multiscale Spatio-Temporal Fusion Network for Video Dehazing
Multiscale Spatio-Temporal Fusion Network for Video Dehazing Open
View article: DirWave: Direction Aware Wavelet-Guided Network for Video Dehazing
DirWave: Direction Aware Wavelet-Guided Network for Video Dehazing Open
View article: Model Tells Itself Where to Attend: Faithfulness Meets Automatic Attention Steering
Model Tells Itself Where to Attend: Faithfulness Meets Automatic Attention Steering Open
Large language models (LLMs) have demonstrated remarkable performance across various real-world tasks. However, they often struggle to fully comprehend and effectively utilize their input contexts, resulting in responses that are unfaithfu…
View article: Robust Reinforcement Learning from Corrupted Human Feedback
Robust Reinforcement Learning from Corrupted Human Feedback Open
Reinforcement learning from human feedback (RLHF) provides a principled framework for aligning AI systems with human preference data. For various reasons, e.g., personal bias, context ambiguity, lack of training, etc, human annotators may …
View article: GEAR: An Efficient KV Cache Compression Recipe for Near-Lossless Generative Inference of LLM
GEAR: An Efficient KV Cache Compression Recipe for Near-Lossless Generative Inference of LLM Open
Key-value (KV) caching has become the de-facto to accelerate generation speed for large language models (LLMs) inference. However, the growing cache demand with increasing sequence length has transformed LLM inference to be a memory bound …
View article: Serum Elabela expression is decreased in hypertensive patients and could be associated with the progression of hypertensive renal damage
Serum Elabela expression is decreased in hypertensive patients and could be associated with the progression of hypertensive renal damage Open
Background Elabela, a recently discovered hormonal peptide containing 32 amino acids, is a ligand for the apelin receptor. It can lower blood pressure and attenuate renal fibrosis. However, the clinicopathological relationship between Elab…
View article: Integration of Instrumental Elements and Vocal Teaching Based on Music Core Literacy
Integration of Instrumental Elements and Vocal Teaching Based on Music Core Literacy Open
The current lack of music core literacy in vocal music teaching needs to be solved, and this paper aims to improve this problem. In the paper, a hybrid attention module is added to the multi-channel of MFCC to extract the acoustic elements…
View article: Tell Your Model Where to Attend: Post-hoc Attention Steering for LLMs
Tell Your Model Where to Attend: Post-hoc Attention Steering for LLMs Open
In human-written articles, we often leverage the subtleties of text style, such as bold and italics, to guide the attention of readers. These textual emphases are vital for the readers to grasp the conveyed information. When interacting wi…
View article: Efficient Long-Range Transformers: You Need to Attend More, but Not Necessarily at Every Layer
Efficient Long-Range Transformers: You Need to Attend More, but Not Necessarily at Every Layer Open
Pretrained transformer models have demonstrated remarkable performance across various natural language processing tasks. These models leverage the attention mechanism to capture long- and short-range dependencies in the sequence. However, …
View article: Serum Elabela expression is decreased in hypertensive patients and could be associated with the progression of hypertensive renal damage
Serum Elabela expression is decreased in hypertensive patients and could be associated with the progression of hypertensive renal damage Open
Background Elabela, a recently discovered hormonal peptide containing 32 amino acids, is a ligand for the apelin receptor. It can lower blood pressure and attenuate renal fibrosis. However, the clinicopathological relationship between the …
View article: Robust Multi-Agent Reinforcement Learning via Adversarial Regularization: Theoretical Foundation and Stable Algorithms
Robust Multi-Agent Reinforcement Learning via Adversarial Regularization: Theoretical Foundation and Stable Algorithms Open
Multi-Agent Reinforcement Learning (MARL) has shown promising results across several domains. Despite this promise, MARL policies often lack robustness and are therefore sensitive to small changes in their environment. This presents a seri…
View article: LoSparse: Structured Compression of Large Language Models based on Low-Rank and Sparse Approximation
LoSparse: Structured Compression of Large Language Models based on Low-Rank and Sparse Approximation Open
Transformer models have achieved remarkable results in various natural language tasks, but they are often prohibitively large, requiring massive memories and computational resources. To reduce the size and complexity of these models, we pr…
View article: AdaLoRA: Adaptive Budget Allocation for Parameter-Efficient Fine-Tuning
AdaLoRA: Adaptive Budget Allocation for Parameter-Efficient Fine-Tuning Open
Fine-tuning large pre-trained language models on downstream tasks has become an important paradigm in NLP. However, common practice fine-tunes all of the parameters in a pre-trained model, which becomes prohibitive when a large number of d…
View article: Efficient Long-Range Transformers: You Need to Attend More, but Not Necessarily at Every Layer
Efficient Long-Range Transformers: You Need to Attend More, but Not Necessarily at Every Layer Open
Pretrained transformer models have demonstrated remarkable performance across various natural language processing tasks. These models leverage the attention mechanism to capture long- and short-range dependencies in the sequence. However, …
View article: Less is More: Task-aware Layer-wise Distillation for Language Model Compression
Less is More: Task-aware Layer-wise Distillation for Language Model Compression Open
Layer-wise distillation is a powerful tool to compress large models (i.e. teacher models) into small ones (i.e., student models). The student distills knowledge from the teacher by mimicking the hidden representations of the teacher at eve…
View article: PLATON: Pruning Large Transformer Models with Upper Confidence Bound of Weight Importance
PLATON: Pruning Large Transformer Models with Upper Confidence Bound of Weight Importance Open
Large Transformer-based models have exhibited superior performance in various natural language processing and computer vision tasks. However, these models contain enormous amounts of parameters, which restrict their deployment to real-worl…
View article: MoEBERT: from BERT to Mixture-of-Experts via Importance-Guided Adaptation
MoEBERT: from BERT to Mixture-of-Experts via Importance-Guided Adaptation Open
Pre-trained language models have demonstrated superior performance in various natural language processing tasks. However, these models usually contain hundreds of millions of parameters, which limits their practicality because of latency r…
View article: Clinical Predictors of Mixed Apneas in Patients with Obstructive Sleep Apnea (OSA)
Clinical Predictors of Mixed Apneas in Patients with Obstructive Sleep Apnea (OSA) Open
Pengfei Liu,1,* Quanhui Chen,1,* Fang Yuan,2 Qingru Zhang,1 Xiaoying Zhang,1 Chan Xue,1 Yuqing Wei,1 Yakun Wang,1 Hanqiao Wang1,2 1Department of Sleep Medicine, The Third Hospital of Hebei Medical University, Shijiazhuang, Hebei, People’s …
View article: New Viewpoint on Readers’ Reception in Formalist Literary Theory
New Viewpoint on Readers’ Reception in Formalist Literary Theory Open
Formalist Literary Theory is an important link in the development process of the research focus of western literary theory from the author to the reader.Formalist Literary Theory advocates the independence and self-sufficiency of literatur…
View article: MoEBERT: from BERT to Mixture-of-Experts via Importance-Guided Adaptation
MoEBERT: from BERT to Mixture-of-Experts via Importance-Guided Adaptation Open
Simiao Zuo, Qingru Zhang, Chen Liang, Pengcheng He, Tuo Zhao, Weizhu Chen. Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2022.
View article: A Biased Graph Neural Network Sampler with Near-Optimal Regret
A Biased Graph Neural Network Sampler with Near-Optimal Regret Open
Graph neural networks (GNN) have recently emerged as a vehicle for applying deep network architectures to graph and relational data. However, given the increasing size of industrial datasets, in many practical situations the message passin…
View article: AdaShift: Decorrelation and Convergence of Adaptive Learning Rate Methods
AdaShift: Decorrelation and Convergence of Adaptive Learning Rate Methods Open
Adam is shown not being able to converge to the optimal solution in certain cases. Researchers recently propose several algorithms to avoid the issue of non-convergence of Adam, but their efficiency turns out to be unsatisfactory in practi…