Parameterized complexity
View article
Deeper, Broader and Artier Domain Generalization Open
The problem of domain generalization is to learn from multiple training domains, and extract a domain-agnostic model that can then be applied to an unseen domain. Domain generalization (DG) has a clear motivation in contexts where there ar…
View article
Meta-Learning for Semi-Supervised Few-Shot Classification Open
In few-shot classification, we are interested in learning algorithms that train a classifier from only a handful of labeled examples. Recent progress in few-shot classification has featured meta-learning, in which a parameterized model for…
View article
PACT: Parameterized Clipping Activation for Quantized Neural Networks Open
Deep learning algorithms achieve high classification accuracy at the expense of significant computation cost. To address this cost, a number of quantization schemes have been proposed - but most of these techniques focused on quantizing we…
View article
Modeling the dynamics of novel coronavirus (2019-nCov) with fractional derivative Open
The present paper describes the mathematical modeling and dynamics of a novel corona virus (2019-nCoV). We describe the brief details of interaction among the bats and unknown hosts, then among the peoples and the infections reservoir (sea…
View article
A Model of Text for Experimentation in the Social Sciences Open
Statistical models of text have become increasingly popular in statistics and computer science as a method of exploring large document collections. Social scientists often want to move beyond exploration, to measurement and experimentation…
View article
Deep forest Open
Current deep-learning models are mostly built upon neural networks, i.e. multiple layers of parameterized differentiable non-linear modules that can be trained by backpropagation. In this paper, we explore the possibility of building deep …
View article
Attention-based Deep Multiple Instance Learning Open
Multiple instance learning (MIL) is a variation of supervised learning where a single class label is assigned to a bag of instances. In this paper, we state the MIL problem as learning the Bernoulli distribution of the bag label where the …
View article
To prune, or not to prune: exploring the efficacy of pruning for model compression Open
Model pruning seeks to induce sparsity in a deep neural network's various connection matrices, thereby reducing the number of nonzero-valued parameters in the model. Recent reports (Han et al., 2015; Narang et al., 2017) prune deep network…
View article
Measuring and Mitigating Unintended Bias in Text Classification Open
We introduce and illustrate a new approach to measuring and mitigating unintended bias in machine learning models. Our definition of unintended bias is parameterized by a test set and a subset of input features. We illustrate how this can …
View article
DropBlock: A regularization method for convolutional networks Open
Deep neural networks often work well when they are over-parameterized and trained with a massive amount of noise and regularization, such as weight decay and dropout. Although dropout is widely used as a regularization technique for fully …
View article
ResNeSt: Split-Attention Networks Open
It is well known that featuremap attention and multi-path representation are important for visual recognition. In this paper, we present a modularized architecture, which applies the channel-wise attention on different network branches to …
View article
Born Again Neural Networks Open
Knowledge Distillation (KD) consists of transferring “knowledge†from one machine learning model (the teacher) to another (the student). Commonly, the teacher is a high-capacity model with formidable performance, while the student is m…
View article
Gradient Descent Provably Optimizes Over-parameterized Neural Networks Open
One of the mysteries in the success of neural networks is randomly initialized first order methods like gradient descent can achieve zero training loss even though the objective function is non-convex and non-smooth. This paper demystifies…
View article
Stronger, Faster and More Explainable: A Graph Convolutional Baseline for Skeleton-based Action Recognition Open
One essential problem in skeleton-based action recognition is how to extract\ndiscriminative features over all skeleton joints. However, the complexity of\nthe State-Of-The-Art (SOTA) models of this task tends to be exceedingly\nsophistica…
View article
A graph-based genetic algorithm and generative model/Monte Carlo tree search for the exploration of chemical space Open
This paper presents a comparison of a graph-based genetic algorithm (GB-GA) and machine learning (ML) results for the optimization of log P values with a constraint for synthetic accessibility and shows that the GA is as good as or better …
View article
On Lazy Training in Differentiable Programming Open
In a series of recent theoretical works, it was shown that strongly over-parameterized neural networks trained with gradient-based methods could converge exponentially fast to zero training loss, with their parameters hardly varying. In th…
View article
Online Knowledge Distillation with Diverse Peers Open
Distillation is an effective knowledge-transfer technique that uses predicted distributions of a powerful teacher model as soft targets to train a less-parameterized student model. A pre-trained high capacity teacher, however, is not alway…
View article
CondConv: Conditionally Parameterized Convolutions for Efficient\n Inference Open
Convolutional layers are one of the basic building blocks of modern deep\nneural networks. One fundamental assumption is that convolutional kernels\nshould be shared for all examples in a dataset. We propose conditionally\nparameterized co…
View article
Federated Meta-Learning with Fast Convergence and Efficient Communication Open
Statistical and systematic challenges in collaboratively training machine learning models across distributed networks of mobile devices have been the bottlenecks in the real-world application of federated learning. In this work, we show th…
View article
Born Again Neural Networks Open
Knowledge Distillation (KD) consists of transferring “knowledge” from one machine learning model (the teacher) to another (the student). Commonly, the teacher is a high-capacity model with formidable performance, while the student is more …
View article
Very High Resolution Object-Based Land Use–Land Cover Urban Classification Using Extreme Gradient Boosting Open
In this letter, the recently developed extreme gradient boosting (Xgboost) classifier is implemented in a very high resolution (VHR) object-based urban land use-land cover application. In detail, we investigated the sensitivity of Xgboost …
View article
Implicit Neural Representations with Periodic Activation Functions Open
Implicitly defined, continuous, differentiable signal representations parameterized by neural networks have emerged as a powerful paradigm, offering many possible benefits over conventional representations. However, current network archite…
View article
Assessing the reliability of species distribution projections in climate change research Open
Aim Forecasting changes in species distribution under future scenarios is one of the most prolific areas of application for species distribution models (SDMs). However, no consensus yet exists on the reliability of such models for drawing …
View article
PremPS: Predicting the impact of missense mutations on protein stability Open
Computational methods that predict protein stability changes induced by missense mutations have made a lot of progress over the past decades. Most of the available methods however have very limited accuracy in predicting stabilizing mutati…
View article
CondConv: Conditionally Parameterized Convolutions for Efficient Inference Open
Convolutional layers are one of the basic building blocks of modern deep neural networks. One fundamental assumption is that convolutional kernels should be shared for all examples in a dataset. We propose conditionally parameterized convo…
View article
Global Convergence of Policy Gradient Methods for the Linear Quadratic Regulator Open
Direct policy gradient methods for reinforcement learning and continuous control problems are a popular approach for a variety of reasons: 1) they are easy to implement without explicit knowledge of the underlying model 2) they are an "end…
View article
DAGs with NO TEARS: Continuous Optimization for Structure Learning Open
Estimating the structure of directed acyclic graphs (DAGs, also known as Bayesian networks) is a challenging problem since the search space of DAGs is combinatorial and scales superexponentially with the number of nodes. Existing approache…
View article
Separating Neural Oscillations from Aperiodic 1/f Activity: Challenges and Recommendations Open
Electrophysiological power spectra typically consist of two components: An aperiodic part usually following an 1/f power law $$P\propto 1/{f}^{\beta }$$ and periodic components appearing as spectral peaks. While the investigation of the pe…
View article
Stochastic Gradient Descent Optimizes Over-parameterized Deep ReLU Networks Open
We study the problem of training deep neural networks with Rectified Linear Unit (ReLU) activation function using gradient descent and stochastic gradient descent. In particular, we study the binary classification problem and show that for…
View article
Differentiable Programming Tensor Networks Open
Differentiable programming is a fresh programming paradigm which composes\nparameterized algorithmic components and trains them using automatic\ndifferentiation (AD). The concept emerges from deep learning but is not only\nlimited to train…