Jeff Pool
YOU?
Author Swipe
View article: Accelerating Sparse Deep Neural Networks
Accelerating Sparse Deep Neural Networks Open
As neural network model sizes have dramatically increased, so has the interest in various techniques to reduce their parameter counts and accelerate their execution. An active area of research in this field is sparsity - encouraging zero v…
View article: Self-Supervised GAN Compression
Self-Supervised GAN Compression Open
Deep learning's success has led to larger and larger models to handle more and more complex tasks; trained models can contain millions of parameters. These large models are compute- and memory-intensive, which makes it a challenge to deplo…
View article: Buddy Compression: Enabling Larger Memory for Deep Learning and HPC Workloads on GPUs
Buddy Compression: Enabling Larger Memory for Deep Learning and HPC Workloads on GPUs Open
GPUs offer orders-of-magnitude higher memory bandwidth than traditional CPU-only systems. However, GPU device memory tends to be relatively small and the memory capacity can not be increased by the user. This paper describes Buddy Compress…
View article: Buddy Compression: Enabling Larger Memory for Deep Learning and HPC\n Workloads on GPUs
Buddy Compression: Enabling Larger Memory for Deep Learning and HPC\n Workloads on GPUs Open
GPUs offer orders-of-magnitude higher memory bandwidth than traditional\nCPU-only systems. However, GPU device memory tends to be relatively small and\nthe memory capacity can not be increased by the user. This paper describes\nBuddy Compr…
View article: Structurally Sparsified Backward Propagation for Faster Long Short-Term Memory Training
Structurally Sparsified Backward Propagation for Faster Long Short-Term Memory Training Open
Exploiting sparsity enables hardware systems to run neural networks faster and more energy-efficiently. However, most prior sparsity-centric optimization techniques only accelerate the forward pass of neural networks and usually require an…
View article: Sparse Persistent RNNs: Squeezing Large Recurrent Networks On-Chip
Sparse Persistent RNNs: Squeezing Large Recurrent Networks On-Chip Open
Recurrent Neural Networks (RNNs) are powerful tools for solving sequence-based problems, but their efficacy and execution time are dependent on the size of the network. Following recent work in simplifying these networks with model pruning…
View article: Efficient Sparse-Winograd Convolutional Neural Networks
Efficient Sparse-Winograd Convolutional Neural Networks Open
Convolutional Neural Networks (CNNs) are computationally intensive, which limits their application on mobile devices. Their energy is dominated by the number of multiplies needed to perform the convolutions. Winograd's minimal filtering al…
View article: Compressing DMA Engine: Leveraging Activation Sparsity for Training Deep Neural Networks
Compressing DMA Engine: Leveraging Activation Sparsity for Training Deep Neural Networks Open
Popular deep learning frameworks require users to fine-tune their memory usage so that the training data of a deep neural network (DNN) fits within the GPU physical memory. Prior work tries to address this restriction by virtualizing the m…
View article: Exploring the Regularity of Sparse Structure in Convolutional Neural Networks
Exploring the Regularity of Sparse Structure in Convolutional Neural Networks Open
Sparsity helps reduce the computational complexity of deep neural networks by skipping zeros. Taking advantage of sparsity is listed as a high priority in next generation DNN accelerators such as TPU. The structure of sparsity, i.e., the g…
View article: DSD: Regularizing Deep Neural Networks with Dense-Sparse-Dense Training Flow.
DSD: Regularizing Deep Neural Networks with Dense-Sparse-Dense Training Flow. Open
Modern deep neural networks have a large number of parameters, making them very powerful machine learning systems. A critical issue for training such large networks on large-scale data-sets is to prevent overfitting while at the same time …
View article: DSD: Dense-Sparse-Dense Training for Deep Neural Networks
DSD: Dense-Sparse-Dense Training for Deep Neural Networks Open
Modern deep neural networks have a large number of parameters, making them very hard to train. We propose DSD, a dense-sparse-dense training flow, for regularizing deep neural networks and achieving better optimization performance. In the …
View article: DSD: Dense-Sparse-Dense Training for Deep Neural Networks
DSD: Dense-Sparse-Dense Training for Deep Neural Networks Open
Modern deep neural networks have a large number of parameters, making them very hard to train. We propose DSD, a dense-sparse-dense training flow, for regularizing deep neural networks and achieving better optimization performance. In the …
View article: Learning both Weights and Connections for Efficient Neural Networks
Learning both Weights and Connections for Efficient Neural Networks Open
Neural networks are both computationally intensive and memory intensive, making them difficult to deploy on embedded systems. Also, conventional networks fix the architecture before training starts; as a result, training cannot improve the…