Xiaocong Du
YOU?
Author Swipe
View article: For the Misgendered Chinese in Gender Bias Research: Multi-Task Learning with Knowledge Distillation for Pinyin Name-Gender Prediction
For the Misgendered Chinese in Gender Bias Research: Multi-Task Learning with Knowledge Distillation for Pinyin Name-Gender Prediction Open
Achieving gender equality is a pivotal factor in realizing the UN's Global Goals for Sustainable Development. Gender bias studies work towards this and rely on name-based gender inference tools to assign individual gender labels when gende…
View article: Layer Compression of Deep Networks with Straight Flows
Layer Compression of Deep Networks with Straight Flows Open
Very deep neural networks lead to significantly better performance on various real tasks. However, it usually causes slow inference and is hard to be deployed on real-world devices. How to reduce the number of layers to save memory and to …
View article: Efficient continual learning at the edge with progressive segmented training
Efficient continual learning at the edge with progressive segmented training Open
There is an increasing need for continual learning in dynamic systems at the edge, such as self-driving vehicles, surveillance drones, and robotic systems. Such a system requires learning from the data stream, training the model to preserv…
View article: Future Gradient Descent for Adapting the Temporal Shifting Data Distribution in Online Recommendation Systems
Future Gradient Descent for Adapting the Temporal Shifting Data Distribution in Online Recommendation Systems Open
One of the key challenges of learning an online recommendation model is the temporal domain shift, which causes the mismatch between the training and testing data distribution and hence domain generalization error. To overcome, we propose …
View article: Exploring Model Stability of Deep Neural Networks for Reliable RRAM-Based In-Memory Acceleration
Exploring Model Stability of Deep Neural Networks for Reliable RRAM-Based In-Memory Acceleration Open
RRAM-based in-memory computing (IMC) effectively accelerates deep neural networks (DNNs). Furthermore, model compression techniques, such as quantization and pruning, are necessary to improve algorithm mapping and hardware performance. How…
View article: Harmless Transfer Learning for Item Embeddings
Harmless Transfer Learning for Item Embeddings Open
Learning embedding layers (for classes, words, items, etc.) is a key component of lots of applications, ranging from natural language processing, recommendation systems to electronic health records, etc.However, the frequency of real-world…
View article: Evolutionary NAS in Light of Model Stability for Accurate Continual Learning
Evolutionary NAS in Light of Model Stability for Accurate Continual Learning Open
Continual learning, the capability to learn new knowledge from streaming data without forgetting the previous knowledge, is a critical requirement for dynamic learning systems, especially for emerging edge devices such as self-driving cars…
View article: Alternate Model Growth and Pruning for Efficient Training of Recommendation Systems
Alternate Model Growth and Pruning for Efficient Training of Recommendation Systems Open
Deep learning recommendation systems at scale have provided remarkable gains through increasing model capacity (i.e. wider and deeper neural networks), but it comes at significant training cost and infrastructure cost. Model pruning is an …
View article: Noise-Based Selection of Robust Inherited Model for Accurate Continual Learning
Noise-Based Selection of Robust Inherited Model for Accurate Continual Learning Open
There is a growing demand for an intelligent system to continually learn knowledge from a data stream. Continual learning requires both the preservation of previous knowledge (i.e., avoiding catastrophic forgetting) and the acquisition of …
View article: Efficient and Modularized Training on FPGA for Real-time Applications
Efficient and Modularized Training on FPGA for Real-time Applications Open
Training of deep Convolution Neural Networks (CNNs) requires a tremendous amount of computation and memory and thus, GPUs are widely used to meet the computation demands of these complex training tasks. However, lacking the flexibility to …
View article: Accurate Inference With Inaccurate RRAM Devices: A Joint Algorithm-Design Solution
Accurate Inference With Inaccurate RRAM Devices: A Joint Algorithm-Design Solution Open
Resistive random access memory (RRAM) is a promising technology for energy-efficient neuromorphic accelerators. However, when a pretrained deep neural network (DNN) model is programmed to an RRAM array for inference, the model suffers from…
View article: Single-Net Continual Learning with Progressive Segmented Training
Single-Net Continual Learning with Progressive Segmented Training Open
There is an increasing need of continual learning in dynamic systems, such as the self-driving vehicle, the surveillance drone, and the robotic system. Such a system requires learning from the data stream, training the model to preserve pr…
View article: Structural Pruning in Deep Neural Networks: A Small-World Approach
Structural Pruning in Deep Neural Networks: A Small-World Approach Open
Deep Neural Networks (DNNs) are usually over-parameterized, causing excessive memory and interconnection cost on the hardware platform. Existing pruning approaches remove secondary parameters at the end of training to reduce the model size…
View article: Efficient Network Construction Through Structural Plasticity
Efficient Network Construction Through Structural Plasticity Open
Deep Neural Networks (DNNs) on hardware is facing excessive computation cost due to the massive number of parameters. A typical training pipeline to mitigate over-parameterization is to pre-define a DNN structure first with redundant learn…
View article: Single-Net Continual Learning with Progressive Segmented Training (PST)
Single-Net Continual Learning with Progressive Segmented Training (PST) Open
There is an increasing need of continual learning in dynamic systems, such as the self-driving vehicle, the surveillance drone, and the robotic system. Such a system requires learning from the data stream, training the model to preserve pr…
View article: Towards Efficient Neural Networks On-a-chip: Joint Hardware-Algorithm Approaches
Towards Efficient Neural Networks On-a-chip: Joint Hardware-Algorithm Approaches Open
Machine learning algorithms have made significant advances in many applications. However, their hardware implementation on the state-of-the-art platforms still faces several challenges and are limited by various factors, such as memory vol…
View article: CGaP: Continuous Growth and Pruning for Efficient Deep Learning
CGaP: Continuous Growth and Pruning for Efficient Deep Learning Open
Today a canonical approach to reduce the computation cost of Deep Neural Networks (DNNs) is to pre-define an over-parameterized model before training to guarantee the learning capacity, and then prune unimportant learning units (filters an…
View article: Programming Exploration of Memristor Crossbar
Programming Exploration of Memristor Crossbar Open
Memristor crossbar is prevailing as one of the most promising candidates to construct the neural network because of their similarity to biological synapses, favorable programmability, simple structure and high performance regarding area ef…