Moin Nabi
YOU?
Author Swipe
View article: Learning Private Representations through Entropy-based Adversarial Training
Learning Private Representations through Entropy-based Adversarial Training Open
How can we learn a representation with high predictive power while preserving user privacy? We present an adversarial representation learning method for sanitizing sensitive content from the learned representation. Specifically, we introdu…
View article: From Dense to Dynamic: Token-Difficulty Driven MoEfication of Pre-Trained LLMs
From Dense to Dynamic: Token-Difficulty Driven MoEfication of Pre-Trained LLMs Open
Training large language models (LLMs) for different inference constraints is computationally expensive, limiting control over efficiency-accuracy trade-offs. Moreover, once trained, these models typically process tokens uniformly, regardle…
View article: Contrastive Perplexity for Controlled Generation: An Application in Detoxifying Large Language Models
Contrastive Perplexity for Controlled Generation: An Application in Detoxifying Large Language Models Open
View article: Multimodal Autoregressive Pre-training of Large Vision Encoders
Multimodal Autoregressive Pre-training of Large Vision Encoders Open
We introduce a novel method for pre-training of large-scale vision encoders. Building on recent advancements in autoregressive pre-training of vision models, we extend this framework to a multimodal setting, i.e., images and text. In this …
View article: SALSA: Soup-based Alignment Learning for Stronger Adaptation in RLHF
SALSA: Soup-based Alignment Learning for Stronger Adaptation in RLHF Open
In Large Language Model (LLM) development, Reinforcement Learning from Human Feedback (RLHF) is crucial for aligning models with human values and preferences. RLHF traditionally relies on the Kullback-Leibler (KL) divergence between the cu…
View article: Computational Bottlenecks of Training Small-scale Large Language Models
Computational Bottlenecks of Training Small-scale Large Language Models Open
While large language models (LLMs) dominate the AI landscape, Small-scale large Language Models (SLMs) are gaining attention due to cost and efficiency demands from consumers. However, there is limited research on the training behavior and…
View article: Chain-of-Sketch: Enabling Global Visual Reasoning
Chain-of-Sketch: Enabling Global Visual Reasoning Open
Modern vision models have achieved remarkable success in benchmarks where local features provide critical information about the target. There is now a growing interest in tackling tasks requiring more global reasoning, where local features…
View article: KV Prediction for Improved Time to First Token
KV Prediction for Improved Time to First Token Open
Inference with transformer-based language models begins with a prompt processing step. In this step, the model generates the first output token and stores the KV cache needed for future generation steps. This prompt processing step can be …
View article: Duo-LLM: A Framework for Studying Adaptive Computation in Large Language Models
Duo-LLM: A Framework for Studying Adaptive Computation in Large Language Models Open
Large Language Models (LLMs) typically generate outputs token by token using a fixed compute budget, leading to inefficient resource utilization. To address this shortcoming, recent advancements in mixture of expert (MoE) models, speculati…
View article: Scaling Smart: Accelerating Large Language Model Pre-training with Small Model Initialization
Scaling Smart: Accelerating Large Language Model Pre-training with Small Model Initialization Open
The pre-training phase of language models often begins with randomly initialized parameters. With the current trends in scaling models, training their large number of parameters can be extremely slow and costly. In contrast, small language…
View article: Contrastive Perplexity for Controlled Generation: An Application in Detoxifying Large Language Models
Contrastive Perplexity for Controlled Generation: An Application in Detoxifying Large Language Models Open
The generation of toxic content by large language models (LLMs) remains a critical challenge for the safe deployment of language technology. We propose a novel framework for implicit knowledge editing and controlled text generation by fine…
View article: A soft nearest-neighbor framework for continual semi-supervised learning
A soft nearest-neighbor framework for continual semi-supervised learning Open
Despite significant advances, the performance of state-of-the-art continual learning approaches hinges on the unrealistic scenario of fully labeled data. In this paper, we tackle this challenge and propose an approach for continual semi-su…
View article: Semi-supervised learning made simple with self-supervised clustering
Semi-supervised learning made simple with self-supervised clustering Open
Self-supervised learning models have been shown to learn rich visual representations without requiring human annotations. However, in many real-world scenarios, labels are partially available, motivating a recent line of work on semi-super…
View article: Semi-supervised learning made simple with self-supervised clustering
Semi-supervised learning made simple with self-supervised clustering Open
Self-supervised learning models have been shown to learn rich visual\nrepresentations without requiring human annotations. However, in many\nreal-world scenarios, labels are partially available, motivating a recent line\nof work on semi-su…
View article: miCSE: Mutual Information Contrastive Learning for Low-shot Sentence Embeddings
miCSE: Mutual Information Contrastive Learning for Low-shot Sentence Embeddings Open
This paper presents miCSE, a mutual information-based contrastive learning framework that significantly advances the state-of-the-art in few-shot sentence embedding.The proposed approach imposes alignment between the attention pattern of d…
View article: miCSE: Mutual Information Contrastive Learning for Low-shot Sentence Embeddings
miCSE: Mutual Information Contrastive Learning for Low-shot Sentence Embeddings Open
This paper presents miCSE, a mutual information-based contrastive learning framework that significantly advances the state-of-the-art in few-shot sentence embedding. The proposed approach imposes alignment between the attention pattern of …
View article: Mixture-of-experts VAEs can disregard variation in surjective multimodal data
Mixture-of-experts VAEs can disregard variation in surjective multimodal data Open
Machine learning systems are often deployed in domains that entail data from multiple modalities, for example, phenotypic and genotypic characteristics describe patients in healthcare. Previous works have developed multimodal variational a…
View article: Uncertainty-Aware Contrastive Distillation for Incremental Semantic Segmentation
Uncertainty-Aware Contrastive Distillation for Incremental Semantic Segmentation Open
A fundamental and challenging problem in deep learning is catastrophic forgetting, i.e., the tendency of neural networks to fail to preserve the knowledge acquired from old tasks when learning new tasks. This problem has been widely invest…
View article: SCD: Self-Contrastive Decorrelation of Sentence Embeddings
SCD: Self-Contrastive Decorrelation of Sentence Embeddings Open
In this paper, we propose Self-Contrastive Decorrelation (SCD), a self-supervised approach. Given an input sentence, it optimizes a joint self-contrastive and decorrelation objective. Learning a representation is facilitated by leveraging …
View article: A Unified Objective for Novel Class Discovery
A Unified Objective for Novel Class Discovery Open
In this paper, we study the problem of Novel Class Discovery (NCD). NCD aims at inferring novel object categories in an unlabeled set by leveraging from prior knowledge of a labeled set containing different, but related classes. Existing a…
View article: Towards Zero-shot Commonsense Reasoning with Self-supervised Refinement of Language Models
Towards Zero-shot Commonsense Reasoning with Self-supervised Refinement of Language Models Open
Can we get existing language models and refine them for zero-shot commonsense reasoning? This paper presents an initial study exploring the feasibility of zero-shot commonsense reasoning for the Winograd Schema Challenge by formulating the…
View article: Attention-based Contrastive Learning for Winograd Schemas
Attention-based Contrastive Learning for Winograd Schemas Open
Self-supervised learning has recently attracted considerable attention in the NLP community for its ability to learn discriminative features using a contrastive objective. This paper investigates whether contrastive learning can be extende…
View article: Towards Zero-shot Commonsense Reasoning with Self-supervised Refinement\n of Language Models
Towards Zero-shot Commonsense Reasoning with Self-supervised Refinement\n of Language Models Open
Can we get existing language models and refine them for zero-shot commonsense\nreasoning? This paper presents an initial study exploring the feasibility of\nzero-shot commonsense reasoning for the Winograd Schema Challenge by\nformulating …
View article: Solo-learn: A Library of Self-supervised Methods for Visual\n Representation Learning
Solo-learn: A Library of Self-supervised Methods for Visual\n Representation Learning Open
This paper presents solo-learn, a library of self-supervised methods for\nvisual representation learning. Implemented in Python, using Pytorch and\nPytorch lightning, the library fits both research and industry needs by\nfeaturing distribu…
View article: Solo-learn: A Library of Self-supervised Methods for Visual Representation Learning
Solo-learn: A Library of Self-supervised Methods for Visual Representation Learning Open
This paper presents solo-learn, a library of self-supervised methods for visual representation learning. Implemented in Python, using Pytorch and Pytorch lightning, the library fits both research and industry needs by featuring distributed…
View article: EaSe: A Diagnostic Tool for VQA based on Answer Diversity
EaSe: A Diagnostic Tool for VQA based on Answer Diversity Open
We propose EASE, a simple diagnostic tool for Visual Question Answering (VQA) which quantifies the difficulty of an image, question sample. EASE is based on the pattern of answers provided by multiple annotators to a given question. In par…
View article: Attention-based Contrastive Learning for Winograd Schemas
Attention-based Contrastive Learning for Winograd Schemas Open
Self-supervised learning has recently attracted considerable attention in the NLP community for its ability to learn discriminative features using a contrastive objective. This paper investigates whether contrastive learning can be extende…
View article: Multimodal Self-supervised Learning for Medical Image Analysis
Multimodal Self-supervised Learning for Medical Image Analysis Open
View article: Multimodal Prototypical Networks for Few-shot Learning
Multimodal Prototypical Networks for Few-shot Learning Open
Although providing exceptional results for many computer vision tasks, state-of-the-art deep learning algorithms catastrophically struggle in low data scenarios. However, if data in additional modalities exist (e.g. text) this can compensa…
View article: Towards Zero-shot Commonsense Reasoning with Self-supervised Refinement of Language Models
Towards Zero-shot Commonsense Reasoning with Self-supervised Refinement of Language Models Open
Can we get existing language models and refine them for zero-shot commonsense reasoning? This paper presents an initial study exploring the feasibility of zero-shot commonsense reasoning for the Winograd Schema Challenge by formulating the…