MNIST database ≈ MNIST database
View article
Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms Open
We present Fashion-MNIST, a new dataset comprising of 28x28 grayscale images of 70,000 fashion products from 10 categories, with 7,000 images per category. The training set has 60,000 images and the test set has 10,000 images. Fashion-MNIS…
View article
On Assessing ML Model Robustness: A Methodological Framework (Academic Track) Open
Due to their uncertainty and vulnerability to adversarial attacks, machine learning (ML) models can lead to severe consequences, including the loss of human life, when embedded in safety-critical systems such as autonomous vehicles. Theref…
View article
Learning Important Features Through Propagating Activation Differences Open
The purported "black box" nature of neural networks is a barrier to adoption in applications where interpretability is essential. Here we present DeepLIFT (Deep Learning Important FeaTures), a method for decomposing the output prediction o…
View article
Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1 Open
We introduce a method to train Binarized Neural Networks (BNNs) - neural networks with binary weights and activations at run-time. At training-time the binary weights and activations are used for computing the parameters gradients. During …
View article
Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering Open
In this work, we are interested in generalizing convolutional neural networks (CNNs) from low-dimensional regular grids, where image, video and speech are represented, to high-dimensional irregular domains, such as social networks, brain c…
View article
Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels Open
Deep neural networks (DNNs) have achieved tremendous success in a variety of applications across many disciplines. Yet, their superior performance comes with the expensive cost of requiring correctly annotated large-scale datasets. Moreove…
View article
Improved Techniques for Training GANs Open
We present a variety of new architectural features and training procedures that we apply to the generative adversarial networks (GANs) framework. We focus on two applications of GANs: semi-supervised learning, and the generation of images …
View article
Explaining nonlinear classification decisions with deep Taylor decomposition Open
S.211-222
View article
The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks Open
Neural network pruning techniques can reduce the parameter counts of trained networks by over 90%, decreasing storage requirements and improving computational performance of inference without compromising accuracy. However, contemporary ex…
View article
InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets Open
This paper describes InfoGAN, an information-theoretic extension to the Generative Adversarial Network that is able to learn disentangled representations in a completely unsupervised manner. InfoGAN is a generative adversarial network that…
View article
Spatio-Temporal Backpropagation for Training High-Performance Spiking Neural Networks Open
Spiking neural networks (SNNs) are promising in ascertaining brain-like behaviors since spikes are capable of encoding spatio-temporal information. Recent schemes, e.g., pre-training from artificial neural networks (ANNs) or direct trainin…
View article
Conversion of Continuous-Valued Deep Networks to Efficient Event-Driven Networks for Image Classification Open
Spiking neural networks (SNNs) can potentially offer an efficient way of doing inference because the neurons in the networks are sparsely activated and computations are event-driven. Previous work showed that simple continuous-valued deep …
View article
Generative Modeling by Estimating Gradients of the Data Distribution Open
We introduce a new generative model where samples are produced via Langevin dynamics using gradients of the data distribution estimated with score matching. Because gradients can be ill-defined and hard to estimate when the data resides on…
View article
Training Deep Spiking Neural Networks Using Backpropagation Open
Deep spiking neural networks (SNNs) hold the potential for improving the latency and energy efficiency of deep neural networks through data-driven event-based computation. However, training such networks is difficult due to the non-differe…
View article
Binarized Neural Networks Open
We introduce a method to train Binarized Neural Networks (BNNs) - neural networks with binary weights and activations at run-time and when computing the parameters' gradient at train-time. We conduct two sets of experiments, each based on …
View article
The Real-World-Weight Cross-Entropy Loss Function: Modeling the Costs of Mislabeling Open
In this paper, we propose a new metric to measure goodness-of-fit for\nclassifiers, the Real World Cost function. This metric factors in information\nabout a real world problem, such as financial impact, that other measures like\naccuracy …
View article
Ternary Weight Networks Open
We present a memory and computation efficient ternary weight networks (TWNs) - with weights constrained to +1, 0 and -1. The Euclidian distance between full (float or double) precision weights and the ternary weights along with a scaling f…
View article
Generating Adversarial Examples with Adversarial Networks Open
Deep neural networks (DNNs) have been found to be vulnerable to adversarial examples resulting from adding small-magnitude perturbations to inputs. Such adversarial examples can mislead DNNs to produce adversary-selected results. Different…
View article
The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks. Open
Neural network pruning techniques can reduce the parameter counts of trained networks by over 90%, decreasing storage requirements and improving computational performance of inference without compromising accuracy. However, contemporary ex…
View article
Provable defenses against adversarial examples via the convex outer adversarial polytope Open
We propose a method to learn deep ReLU-based classifiers that are provably robust against norm-bounded adversarial perturbations on the training data. For previously unseen examples, the approach is guaranteed to detect all adversarial exa…
View article
Attention-based Deep Multiple Instance Learning Open
Multiple instance learning (MIL) is a variation of supervised learning where a single class label is assigned to a bag of instances. In this paper, we state the MIL problem as learning the Bernoulli distribution of the bag label where the …
View article
An Analysis Of Convolutional Neural Networks For Image Classification Open
This paper presents an empirical analysis of theperformance of popular convolutional neural networks (CNNs) for identifying objects in real time video feeds. The most popular convolution neural networks for object detection and object cate…
View article
PathNet: Evolution Channels Gradient Descent in Super Neural Networks Open
For artificial general intelligence (AGI) it would be efficient if multiple users trained the same giant neural network, permitting parameter reuse, without catastrophic forgetting. PathNet is a first step in this direction. It is a neural…
View article
Direct Training for Spiking Neural Networks: Faster, Larger, Better Open
Spiking neural networks (SNNs) that enables energy efficient implementation on emerging neuromorphic hardware are gaining more attention. Yet now, SNNs have not shown competitive performance compared with artificial neural networks (ANNs),…
View article
Group Equivariant Convolutional Networks Open
We introduce Group equivariant Convolutional Neural Networks (G-CNNs), a natural generalization of convolutional neural networks that reduces sample complexity by exploiting symmetries. G-CNNs use G-convolutions, a new type of layer that e…
View article
Deep Bayesian Active Learning with Image Data Open
Even though active learning forms an important pillar of machine learning, deep learning tools are not prevalent within it. Deep learning poses several difficulties when used in an active learning setting. First, active learning (AL) metho…
View article
EAD: Elastic-Net Attacks to Deep Neural Networks via Adversarial Examples Open
Recent studies have highlighted the vulnerability of deep neural networks (DNNs) to adversarial examples — a visually indistinguishable adversarial image can easily be crafted to cause a well-trained model to misclassify. Existing methods …
View article
Three scenarios for continual learning Open
Standard artificial neural networks suffer from the well-known issue of catastrophic forgetting, making continual or lifelong learning difficult for machine learning. In recent years, numerous methods have been proposed for continual learn…
View article
Advancing Neuromorphic Computing With Loihi: A Survey of Results and Outlook Open
Deep artificial neural networks apply principles of the brain's information processing that led to breakthroughs in machine learning spanning many problem domains. Neuromorphic computing aims to take this a step further to chips more direc…
View article
Measuring Catastrophic Forgetting in Neural Networks Open
Deep neural networks are used in many state-of-the-art systems for machine perception. Once a network is trained to do a specific task, e.g., bird classification, it cannot easily be trained to do new tasks, e.g., incrementally learning to…