Qixuan Feng
YOU?
Author Swipe
View article: DiPaCo: Distributed Path Composition
DiPaCo: Distributed Path Composition Open
Progress in machine learning (ML) has been fueled by scaling neural network models. This scaling has been enabled by ever more heroic feats of engineering, necessary for accommodating ML approaches that require high bandwidth communication…
View article: Continual Learning via Sequential Function-Space Variational Inference
Continual Learning via Sequential Function-Space Variational Inference Open
Sequential Bayesian inference over predictive functions is a natural framework for continual learning from streams of data. However, applying it to neural networks has proved challenging in practice. Addressing the drawbacks of existing te…
View article: DiLoCo: Distributed Low-Communication Training of Language Models
DiLoCo: Distributed Low-Communication Training of Language Models Open
Large language models (LLM) have become a critical component in many applications of machine learning. However, standard approaches to training LLM require a large number of tightly interconnected accelerators, with devices exchanging grad…
View article: Benchmarking Bayesian Deep Learning on Diabetic Retinopathy Detection Tasks
Benchmarking Bayesian Deep Learning on Diabetic Retinopathy Detection Tasks Open
Bayesian deep learning seeks to equip deep neural networks with the ability to precisely quantify their predictive uncertainty, and has promised to make deep learning more reliable for safety-critical real-world applications. Yet, existing…
View article: Uncertainty Baselines: Benchmarks for Uncertainty & Robustness in Deep Learning
Uncertainty Baselines: Benchmarks for Uncertainty & Robustness in Deep Learning Open
High-quality estimates of uncertainty and robustness are crucial for numerous real-world applications, especially for deep learning which underlies many deployed ML systems. The ability to compare techniques for improving these estimates i…