Chuanxiong Guo
YOU?
Author Swipe
View article: Efficient LLM Serving on Hybrid Real-time and Best-effort Requests
Efficient LLM Serving on Hybrid Real-time and Best-effort Requests Open
Recent breakthroughs in large Language Models (LLMs) have enabled various generative tasks on a single model. Real-world services (e.g., OpenAI's ChatGPT [27]) powered by an LLM often concurrently support latency-critical requests for inte…
View article: Collie: Finding Performance Anomalies in RDMA Subsystems
Collie: Finding Performance Anomalies in RDMA Subsystems Open
High-speed RDMA networks are getting rapidly adopted in the industry for their low latency and reduced CPU overheads. To verify that RDMA can be used in production, system administrators need to understand the set of application workloads …
View article: dPRO: A Generic Profiling and Optimization System for Expediting Distributed DNN Training
dPRO: A Generic Profiling and Optimization System for Expediting Distributed DNN Training Open
Distributed training using multiple devices (e.g., GPUs) has been widely adopted for learning DNN models over large datasets. However, the performance of large-scale distributed training tends to be far from linear speed-up in practice. Gi…
View article: Aryl: An Elastic Cluster Scheduler for Deep Learning
Aryl: An Elastic Cluster Scheduler for Deep Learning Open
Companies build separate training and inference GPU clusters for deep learning, and use separate schedulers to manage them. This leads to problems for both training and inference: inference clusters have low GPU utilization when the traffi…
View article: Prediction of GPU Failures Under Deep Learning Workloads
Prediction of GPU Failures Under Deep Learning Workloads Open
Graphics processing units (GPUs) are the de facto standard for processing deep learning (DL) tasks. Meanwhile, GPU failures, which are inevitable, cause severe consequences in DL tasks: they disrupt distributed trainings, crash inference s…
View article: BGL: GPU-Efficient GNN Training by Optimizing Graph Data I/O and Preprocessing
BGL: GPU-Efficient GNN Training by Optimizing Graph Data I/O and Preprocessing Open
Graph neural networks (GNNs) have extended the success of deep neural networks (DNNs) to non-Euclidean graph data, achieving ground-breaking performance on various tasks such as node classification and graph property prediction. Nonetheles…
View article: Serving DNN Models with Multi-Instance GPUs: A Case of the Reconfigurable Machine Scheduling Problem
Serving DNN Models with Multi-Instance GPUs: A Case of the Reconfigurable Machine Scheduling Problem Open
Multi-Instance GPU (MIG) is a new feature introduced by NVIDIA A100 GPUs that partitions one physical GPU into multiple GPU instances. With MIG, A100 can be the most cost-efficient GPU ever for serving Deep Neural Networks (DNNs). However,…
View article: AutoLRS: Automatic Learning-Rate Schedule by Bayesian Optimization on the Fly
AutoLRS: Automatic Learning-Rate Schedule by Bayesian Optimization on the Fly Open
Gradient compression is a widely-established remedy to tackle the communication bottleneck in distributed training of large deep neural networks (DNNs). Under the error-feedback framework, Top-k sparsification, sometimes with k as little a…
View article: AutoLRS: Automatic Learning-Rate Schedule by Bayesian Optimization on the Fly
AutoLRS: Automatic Learning-Rate Schedule by Bayesian Optimization on the Fly Open
The learning rate (LR) schedule is one of the most important hyper-parameters needing careful tuning in training DNNs. However, it is also one of the least automated parts of machine learning systems and usually costs significant manual ef…
View article: Tersecades: efficient data compression in stream processing
Tersecades: efficient data compression in stream processing Open
This work is the first systematic investigation of stream processing with data compression: we have not only identified a set of factors that influence the benefits and overheads of compression, but have also demonstrated that compression …