Gauri Joshi
YOU?
Author Swipe
View article: Adaptive Federated Learning via Dynamical System Model
Adaptive Federated Learning via Dynamical System Model Open
Hyperparameter selection is critical for stable and efficient convergence of heterogeneous federated learning, where clients differ in computational capabilities, and data distributions are non-IID. Tuning hyperparameters is a manual and c…
View article: Language Model Planning from an Information Theoretic Perspective
Language Model Planning from an Information Theoretic Perspective Open
The extent to which decoder-only language models (LMs) engage in planning, that is, organizing intermediate computations to support coherent long-range generation, remains an open and important question, with implications for interpretabil…
View article: Natural Policy Gradient for Average Reward Non-Stationary RL
Natural Policy Gradient for Average Reward Non-Stationary RL Open
We consider the problem of non-stationary reinforcement learning (RL) in the infinite-horizon average-reward setting. We model it by a Markov Decision Process with time-varying rewards and transition probabilities, with a variation budget …
View article: The Cost of Shuffling in Private Gradient Based Optimization
The Cost of Shuffling in Private Gradient Based Optimization Open
We consider the problem of differentially private (DP) convex empirical risk minimization (ERM). While the standard DP-SGD algorithm is theoretically well-established, practical implementations often rely on shuffled gradient methods that …
View article: Monochromatic arithmetic progressions in the Fibonacci, Thue-Morse, and Rudin-Shapiro words
Monochromatic arithmetic progressions in the Fibonacci, Thue-Morse, and Rudin-Shapiro words Open
We investigate the lengths and starting positions of the longest monochromatic arithmetic progressions for a fixed difference in the Fibonacci word. We provide a complete classification for their lengths in terms of a simple formula. Our s…
View article: Formulation and Evaluation of Lip Balm
Formulation and Evaluation of Lip Balm Open
View article: Optimized Tradeoffs for Private Prediction with Majority Ensembling
Optimized Tradeoffs for Private Prediction with Majority Ensembling Open
We study a classical problem in private prediction, the problem of computing an $(mε, δ)$-differentially private majority of $K$ $(ε, Δ)$-differentially private algorithms for $1 \leq m \leq K$ and $1 > δ\geq Δ\geq 0$. Standard methods suc…
View article: Nonlinear Stochastic Gradient Descent and Heavy-tailed Noise: A Unified Framework and High-probability Guarantees
Nonlinear Stochastic Gradient Descent and Heavy-tailed Noise: A Unified Framework and High-probability Guarantees Open
We study high-probability convergence in online learning, in the presence of heavy-tailed noise. To combat the heavy tails, a general framework of nonlinear SGD methods is considered, subsuming several popular nonlinearities like sign, qua…
View article: FedECADO: A Dynamical System Model of Federated Learning
FedECADO: A Dynamical System Model of Federated Learning Open
Federated learning harnesses the power of distributed optimization to train a unified machine learning model across separate clients. However, heterogeneous data distributions and computational workloads can lead to inconsistent updates an…
View article: Erasure Coded Neural Network Inference via Fisher Averaging
Erasure Coded Neural Network Inference via Fisher Averaging Open
Erasure-coded computing has been successfully used in cloud systems to reduce tail latency caused by factors such as straggling servers and heterogeneous traffic variations. A majority of cloud computing traffic now consists of inference o…
View article: FedFisher: Leveraging Fisher Information for One-Shot Federated Learning
FedFisher: Leveraging Fisher Information for One-Shot Federated Learning Open
Standard federated learning (FL) algorithms typically require multiple rounds of communication between the server and the clients, which has several drawbacks, including requiring constant network connectivity, repeated investment of compu…
View article: Federated Offline Reinforcement Learning: Collaborative Single-Policy Coverage Suffices
Federated Offline Reinforcement Learning: Collaborative Single-Policy Coverage Suffices Open
Offline reinforcement learning (RL), which seeks to learn an optimal policy using offline data, has garnered significant interest due to its potential in critical applications where online data collection is infeasible or expensive. This w…
View article: Efficient Reinforcement Learning for Routing Jobs in Heterogeneous Queueing Systems
Efficient Reinforcement Learning for Routing Jobs in Heterogeneous Queueing Systems Open
We consider the problem of efficiently routing jobs that arrive into a central queue to a system of heterogeneous servers. Unlike homogeneous systems, a threshold policy, that routes jobs to the slow server(s) when the queue length exceeds…
View article: Heterogeneous LoRA for Federated Fine-tuning of On-Device Foundation Models
Heterogeneous LoRA for Federated Fine-tuning of On-Device Foundation Models Open
Foundation models (FMs) adapt well to specific domains or tasks with fine-tuning, and federated learning (FL) enables the potential for privacy-preserving fine-tuning of the FMs with on-device local data. For federated fine-tuning of FMs, …
View article: Correlation Aware Sparsified Mean Estimation Using Random Projection
Correlation Aware Sparsified Mean Estimation Using Random Projection Open
We study the problem of communication-efficient distributed vector mean estimation, a commonly used subroutine in distributed optimization and Federated Learning (FL). Rand-$k$ sparsification is a commonly used technique to reduce communic…
View article: High-probability Convergence Bounds for Nonlinear Stochastic Gradient Descent Under Heavy-tailed Noise
High-probability Convergence Bounds for Nonlinear Stochastic Gradient Descent Under Heavy-tailed Noise Open
We study high-probability convergence guarantees of learning on streaming data in the presence of heavy-tailed noise. In the proposed scenario, the model is updated in an online fashion, as new information is observed, without storing any …
View article: Local or Global: Selective Knowledge Assimilation for Federated Learning with Limited Labels
Local or Global: Selective Knowledge Assimilation for Federated Learning with Limited Labels Open
Many existing FL methods assume clients with fully-labeled data, while in realistic settings, clients have limited labels due to the expensive and laborious process of labeling. Limited labeled local data of the clients often leads to thei…
View article: The Blessing of Heterogeneity in Federated Q-Learning: Linear Speedup and Beyond
The Blessing of Heterogeneity in Federated Q-Learning: Linear Speedup and Beyond Open
When the data used for reinforcement learning (RL) are collected by multiple agents in a distributed manner, federated versions of RL algorithms allow collaborative learning without the need for agents to share their local data. In this pa…
View article: Federated Minimax Optimization with Client Heterogeneity
Federated Minimax Optimization with Client Heterogeneity Open
Minimax optimization has seen a surge in interest with the advent of modern applications such as GANs, and it is inherently more challenging than simple minimization. The difficulty is exacerbated by the training data residing at multiple …
View article: On the Convergence of Federated Averaging with Cyclic Client Participation
On the Convergence of Federated Averaging with Cyclic Client Participation Open
Federated Averaging (FedAvg) and its variants are the most popular optimization algorithms in federated learning (FL). Previous convergence analyses of FedAvg either assume full client participation or partial client participation where th…
View article: FedExP: Speeding Up Federated Averaging via Extrapolation
FedExP: Speeding Up Federated Averaging via Extrapolation Open
Federated Averaging (FedAvg) remains the most popular algorithm for Federated Learning (FL) optimization due to its simple implementation, stateless nature, and privacy guarantees combined with secure aggregation. Recent work has sought to…
View article: Training ERP for Student Placement
Training ERP for Student Placement Open
The management of Training and Placement is supported by paper-based systems, databases, spreadsheets and E-mail communications. The aim of this project is Automation of Training and Placement unit of College or academic institutes. This i…
View article: Correlated combinatorial bandits for online resource allocation
Correlated combinatorial bandits for online resource allocation Open
We study a sequential resource allocation problem where, at each round, the decision-maker needs to allocate its limited budget among different available entities. In doing so, the decision-maker obtains the reward for each entity in that …
View article: Tackling heterogeneous traffic in multi-access systems via erasure coded servers
Tackling heterogeneous traffic in multi-access systems via erasure coded servers Open
Most data generated by modern applications is stored in the cloud, and there is an exponential growth in the volume of jobs to access these data and perform computations using them. The volume of data access or computing jobs can be hetero…
View article: FedVARP: Tackling the Variance Due to Partial Client Participation in Federated Learning
FedVARP: Tackling the Variance Due to Partial Client Participation in Federated Learning Open
Data-heterogeneous federated learning (FL) systems suffer from two significant sources of convergence error: 1) client drift error caused by performing multiple local optimization steps at clients, and 2) partial client participation error…
View article: Tackling Heterogeneous Traffic in Multi-access Systems via Erasure Coded Servers
Tackling Heterogeneous Traffic in Multi-access Systems via Erasure Coded Servers Open
Most data generated by modern applications is stored in the cloud, and there is an exponential growth in the volume of jobs to access these data and perform computations using them. The volume of data access or computing jobs can be hetero…
View article: Federated Stochastic Approximation under Markov Noise and Heterogeneity: Applications in Reinforcement Learning
Federated Stochastic Approximation under Markov Noise and Heterogeneity: Applications in Reinforcement Learning Open
Since reinforcement learning algorithms are notoriously data-intensive, the task of sampling observations from the environment is usually split across multiple agents. However, transferring these observations from the agents to a central l…
View article: On the Unreasonable Effectiveness of Federated Averaging with Heterogeneous Data
On the Unreasonable Effectiveness of Federated Averaging with Heterogeneous Data Open
Existing theory predicts that data heterogeneity will degrade the performance of the Federated Averaging (FedAvg) algorithm in federated learning. However, in practice, the simple FedAvg algorithm converges very well. This paper explains t…
View article: Federated Learning under Distributed Concept Drift
Federated Learning under Distributed Concept Drift Open
Federated Learning (FL) under distributed concept drift is a largely unexplored area. Although concept drift is itself a well-studied phenomenon, it poses particular challenges for FL, because drifts arise staggered in time and space (acro…
View article: Maximizing Global Model Appeal in Federated Learning
Maximizing Global Model Appeal in Federated Learning Open
Federated learning typically considers collaboratively training a global model using local data at edge clients. Clients may have their own individual requirements, such as having a minimal training loss threshold, which they expect to be …