Chuan Wu
YOU?
Author Swipe
View article: DCP: Addressing Input Dynamism In Long-Context Training via Dynamic Context Parallelism
DCP: Addressing Input Dynamism In Long-Context Training via Dynamic Context Parallelism Open
Context parallelism has emerged as a key technique to support long-context training, a growing trend in generative AI for modern large models. However, existing context parallel methods rely on static parallelization configurations that ov…
View article: Robust LLM Training Infrastructure at ByteDance
Robust LLM Training Infrastructure at ByteDance Open
View article: Leadless pacemakers: A review of communication methods, energy management, and clinical applications
Leadless pacemakers: A review of communication methods, energy management, and clinical applications Open
Leadless pacemakers have emerged as a mainstream clinical solution, and their communication capabilities, crucial for reliable pacing and device monitoring, continue to evolve. This review systematically examines the fundamental principles…
View article: On the Interplay between Graph Structure and Learning Algorithms in Graph Neural Networks
On the Interplay between Graph Structure and Learning Algorithms in Graph Neural Networks Open
This paper studies the interplay between learning algorithms and graph structure for graph neural networks (GNNs). Existing theoretical studies on the learning dynamics of GNNs primarily focus on the convergence rates of learning algorithm…
View article: Microbial Corrosion Behavior of L245 Pipeline Steel in the Presence of Iron-Oxidizing Bacteria and Shewanella algae
Microbial Corrosion Behavior of L245 Pipeline Steel in the Presence of Iron-Oxidizing Bacteria and Shewanella algae Open
Microbiologically influenced corrosion (MIC) poses significant challenges in oilfield water injection environments, leading to substantial socioeconomic losses. L245 steel, a low-alloy steel widely used in oil and gas pipelines due to its …
View article: HybridFlow: A Flexible and Efficient RLHF Framework
HybridFlow: A Flexible and Efficient RLHF Framework Open
Reinforcement Learning from Human Feedback (RLHF) is widely used in Large\nLanguage Model (LLM) alignment. Traditional RL can be modeled as a dataflow,\nwhere each node represents computation of a neural network (NN) and each edge\ndenotes…
View article: Mitigating Unfairness in Differentially-Private Federated Learning
Mitigating Unfairness in Differentially-Private Federated Learning Open
Federated learning is a new learning paradigm which utilizes crowdsourced data stored at dispersed user devices (aka clients) to learn a global model. Studies have shown that even though data are kept on local devices, an adversary is stil…
View article: Developing and Utilizing a Large-Scale Cantonese Dataset for Multi-Tasking in Large Language Models
Developing and Utilizing a Large-Scale Cantonese Dataset for Multi-Tasking in Large Language Models Open
High-quality data resources play a crucial role in learning large language models (LLMs), particularly for low-resource languages like Cantonese. Despite having more than 85 million native speakers, Cantonese is still considered a low-reso…
View article: A machine-learning approach to optimize nutritional properties and organic wastes recycling efficiency conversed by black soldier fly (Hermetia illucens)
A machine-learning approach to optimize nutritional properties and organic wastes recycling efficiency conversed by black soldier fly (Hermetia illucens) Open
Suboptimal nutrition in organic waste limits the growth of black soldier fly (BSF) larvae, thereby reducing biowaste recycling efficiency. In this study, weight gain data from BSF larvae fed diets with distinct nutrient compositions were u…
View article: Developing and Utilizing a Large-Scale Cantonese Dataset for Multi-Tasking in Large Language Models
Developing and Utilizing a Large-Scale Cantonese Dataset for Multi-Tasking in Large Language Models Open
View article: ProReason: Multi-Modal Proactive Reasoning with Decoupled Eyesight and Wisdom
ProReason: Multi-Modal Proactive Reasoning with Decoupled Eyesight and Wisdom Open
View article: How Well Do LLMs Handle Cantonese? Benchmarking Cantonese Capabilities of Large Language Models
How Well Do LLMs Handle Cantonese? Benchmarking Cantonese Capabilities of Large Language Models Open
View article: Echo: Simulating Distributed Training At Scale
Echo: Simulating Distributed Training At Scale Open
Simulation offers unique values for both enumeration and extrapolation purposes, and is becoming increasingly important for managing the massive machine learning (ML) clusters and large-scale distributed training jobs. In this paper, we bu…
View article: Metastable pitting corrosion behavior of the Incoloy 825 liner of metallurgically clad pipe in simulated oilfield produced water
Metastable pitting corrosion behavior of the Incoloy 825 liner of metallurgically clad pipe in simulated oilfield produced water Open
This study investigates the metastable pitting corrosion behavior and passive film characteristics of metallurgically clad pipe (MCP) 825 within simulated oilfield produced water. Electrochemical testing and microstructural examination wer…
View article: How Well Do LLMs Handle Cantonese? Benchmarking Cantonese Capabilities of Large Language Models
How Well Do LLMs Handle Cantonese? Benchmarking Cantonese Capabilities of Large Language Models Open
The rapid evolution of large language models (LLMs) has transformed the competitive landscape in natural language processing (NLP), particularly for English and other data-rich languages. However, underrepresented languages like Cantonese,…
View article: Heta: Distributed Training of Heterogeneous Graph Neural Networks
Heta: Distributed Training of Heterogeneous Graph Neural Networks Open
Heterogeneous Graph Neural Networks (HGNNs) leverage diverse semantic relationships in Heterogeneous Graphs (HetGs) and have demonstrated remarkable learning performance in various applications. However, current distributed GNN training sy…
View article: Data Augmentation of Multi-turn Psychological Dialogue via Knowledge-driven Progressive Thought Prompting
Data Augmentation of Multi-turn Psychological Dialogue via Knowledge-driven Progressive Thought Prompting Open
Existing dialogue data augmentation (DA) techniques predominantly focus on augmenting utterance-level dialogues, which makes it difficult to take dialogue contextual information into account. The advent of large language models (LLMs) has …
View article: Lancet: Accelerating Mixture-of-Experts Training via Whole Graph Computation-Communication Overlapping
Lancet: Accelerating Mixture-of-Experts Training via Whole Graph Computation-Communication Overlapping Open
The Mixture-of-Expert (MoE) technique plays a crucial role in expanding the size of DNN model parameters. However, it faces the challenge of extended all-to-all communication latency during the training process. Existing methods attempt to…
View article: Preface: Heavy metal(loid)s at mining & metallurgical sites: Fate, risk and remediation
Preface: Heavy metal(loid)s at mining & metallurgical sites: Fate, risk and remediation Open
View article: BG-HGNN: Toward Scalable and Efficient Heterogeneous Graph Neural Network
BG-HGNN: Toward Scalable and Efficient Heterogeneous Graph Neural Network Open
Many computer vision and machine learning problems are modelled as learning tasks on heterogeneous graphs, featuring a wide array of relations from diverse types of nodes and edges. Heterogeneous graph neural networks (HGNNs) stand out as …
View article: On the Topology Awareness and Generalization Performance of Graph Neural Networks
On the Topology Awareness and Generalization Performance of Graph Neural Networks Open
Many computer vision and machine learning problems are modelled as learning tasks on graphs where graph neural networks GNNs have emerged as a dominant tool for learning representations of graph structured data A key feature of GNNs is the…
View article: LLM-PQ: Serving LLM on Heterogeneous Clusters with Phase-Aware Partition and Adaptive Quantization
LLM-PQ: Serving LLM on Heterogeneous Clusters with Phase-Aware Partition and Adaptive Quantization Open
Recent breakthroughs in Large-scale language models (LLMs) have demonstrated impressive performance on various tasks. The immense sizes of LLMs have led to very high resource demand and cost for running the models. Though the models are la…
View article: Towards Robust Graph Incremental Learning on Evolving Graphs
Towards Robust Graph Incremental Learning on Evolving Graphs Open
Incremental learning is a machine learning approach that involves training a model on a sequence of tasks, rather than all tasks at once. This ability to learn incrementally from a stream of tasks is crucial for many real-world application…
View article: HAP: SPMD DNN Training on Heterogeneous GPU Clusters with Automated Program Synthesis
HAP: SPMD DNN Training on Heterogeneous GPU Clusters with Automated Program Synthesis Open
Single-Program-Multiple-Data (SPMD) parallelism has recently been adopted to train large deep neural networks (DNNs). Few studies have explored its applicability on heterogeneous clusters, to fully exploit available resources for large mod…
View article: Identifying Frailty in Older Adults Receiving Home Care Assessment Using Machine Learning: Longitudinal Observational Study on the Role of Classifier, Feature Selection, and Sample Size
Identifying Frailty in Older Adults Receiving Home Care Assessment Using Machine Learning: Longitudinal Observational Study on the Role of Classifier, Feature Selection, and Sample Size Open
Background Machine learning techniques are starting to be used in various health care data sets to identify frail persons who may benefit from interventions. However, evidence about the performance of machine learning techniques compared t…
View article: Microstructural Evolution and Mechanical Properties of Ti6al4v Alloy Manufactured by the Multi-Pass Hot Caliber Rolling at 700℃ and 800℃ with Different Reductions
Microstructural Evolution and Mechanical Properties of Ti6al4v Alloy Manufactured by the Multi-Pass Hot Caliber Rolling at 700℃ and 800℃ with Different Reductions Open
View article: A Multi-Criteria Decision Support Model for New Energy Vehicle Selection Considering Social Media Influencer Reviews and Personalized Preferences
A Multi-Criteria Decision Support Model for New Energy Vehicle Selection Considering Social Media Influencer Reviews and Personalized Preferences Open
View article: GNNFlow: A Distributed Framework for Continuous Temporal GNN Learning on Dynamic Graphs
GNNFlow: A Distributed Framework for Continuous Temporal GNN Learning on Dynamic Graphs Open
Graph Neural Networks (GNNs) play a crucial role in various fields. However, most existing deep graph learning frameworks assume pre-stored static graphs and do not support training on graph streams. In contrast, many real-world graphs are…
View article: DynaPipe: Optimizing Multi-task Training through Dynamic Pipelines
DynaPipe: Optimizing Multi-task Training through Dynamic Pipelines Open
Multi-task model training has been adopted to enable a single deep neural network model (often a large language model) to handle multiple tasks (e.g., question answering and text summarization). Multi-task training commonly receives input …
View article: CDMPP: A Device-Model Agnostic Framework for Latency Prediction of Tensor Programs
CDMPP: A Device-Model Agnostic Framework for Latency Prediction of Tensor Programs Open
Deep Neural Networks (DNNs) have shown excellent performance in a wide range of machine learning applications. Knowing the latency of running a DNN model or tensor program on a specific device is useful in various tasks, such as DNN graph-…