Approximate inference
View article
Variational Inference: A Review for Statisticians Open
One of the core problems of modern statistics is to approximate difficult-to-compute probability densities. This problem is especially important in Bayesian statistics, which frames all inference about unknown quantities as a calculation i…
View article
Bayesian Computing with INLA: A Review Open
The key operation in Bayesian inference is to compute high-dimensional integrals. An old approximate technique is the Laplace method or approximation, which dates back to Pierre-Simon Laplace (1774). This simple idea approximates the integ…
View article
The Nonparanormal: Semiparametric Estimation of High Dimensional Undirected Graphs Open
Recent methods for estimating sparse undirected graphs for real-valued data in high dimensional problems rely heavily on the assumption of normality. We show how to use a semiparametric Gaussian copula---or "nonparanormal"---for high dimen…
View article
Advances in Variational Inference Open
Many modern unsupervised or semi-supervised machine learning algorithms rely on Bayesian probabilistic models. These models are usually intractable and thus require approximate inference. Variational inference (VI) lets us approximate a hi…
View article
Estimating individual treatment effect: generalization bounds and algorithms Open
There is intense interest in applying machine learning to problems of causal inference in fields such as healthcare, economics and education. In particular, individual-level causal inference has important applications such as precision med…
View article
Dropout Inference in Bayesian Neural Networks with Alpha-divergences Open
To obtain uncertainty estimates with real-world Bayesian deep learning models, practical inference approximations are needed. Dropout variational inference (VI) for example has been used for machine vision and medical applications, but VI …
View article
Deep Gaussian Processes for Regression using Approximate Expectation Propagation Open
Deep Gaussian processes (DGPs) are multi-layer hierarchical generalisations of Gaussian processes (GPs) and are formally equivalent to neural networks with multiple, infinitely wide hidden layers. DGPs are nonparametric probabilistic model…
View article
Inference Suboptimality in Variational Autoencoders Open
Amortized inference allows latent-variable models trained via variational learning to scale to large datasets. The quality of approximate inference is determined by two factors: a) the capacity of the variational distribution to match the …
View article
Deep Probabilistic Programming Open
We propose Edward, a Turing-complete probabilistic programming language. Edward defines two compositional representations---random variables and inference. By treating inference as a first class citizen, on a par with modeling, we show tha…
View article
Accelerating Asymptotically Exact MCMC for Computationally Intensive Models via Local Approximations Open
We construct a new framework for accelerating Markov chain Monte Carlo in posterior sampling problems where standard methods are limited by the computational cost of the likelihood, or of numerical models embedded therein. Our approach int…
View article
TerpreT: A Probabilistic Programming Language for Program Induction Open
We study machine learning formulations of inductive program synthesis; given input-output examples, we try to synthesize source code that maps inputs to corresponding outputs. Our aims are to develop new machine learning approaches based o…
View article
A Unifying Framework for Gaussian Process Pseudo-Point Approximations using Power Expectation Propagation Open
Gaussian processes (GPs) are flexible distributions over functions that enable high-level assumptions about unknown functions to be encoded in a parsimonious, flexible and general way. Although elegant, the application of GPs is limited by…
View article
The Manhattan Frame Model—Manhattan World Inference in the Space of Surface Normals Open
Objects and structures within man-made environments typically exhibit a high degree of organization in the form of orthogonal and parallel planes. Traditional approaches utilize these regularities via the restrictive, and rather local, Man…
View article
Accelerating DNN Inference With Reliability Guarantee in Vehicular Edge Computing Open
This paper explores on accelerating Deep Neural Network (DNN) inference with reliability guarantee in Vehicular Edge Computing (VEC) by considering the synergistic impacts of vehicle mobility and Vehicle-to-Vehicle/Infrastructure (V2V/V2I)…
View article
Inference Networks for Sequential Monte Carlo in Graphical Models Open
We introduce a new approach for amortizing inference in directed graphical models by learning heuristic approximations to stochastic inverses, designed specifically for use as proposal distributions in sequential Monte Carlo methods. We de…
View article
Expectation propagation as a way of life Open
A common divide-and-conquer approach for Bayesian computation with big data is to partition the data, perform local inference for each piece separately, and combine the results to obtain a global posterior approximation. While being concep…
View article
Importance Weighting and Variational Inference Open
Recent work used importance sampling ideas for better variational bounds on likelihoods. We clarify the applicability of these ideas to pure probabilistic inference, by showing the resulting Importance Weighted Variational Inference (IWVI)…
View article
A factor graph approach to automated design of Bayesian signal processing algorithms Open
The benefits of automating design cycles for Bayesian inference-based algorithms are becoming increasingly recognized by the machine learning community. As a result, interest in probabilistic programming frameworks has much increased over …
View article
VariBAD: A Very Good Method for Bayes-Adaptive Deep RL via Meta-Learning Open
Trading off exploration and exploitation in an unknown environment is key to maximising expected return during learning. A Bayes-optimal policy, which does so optimally, conditions its actions not only on the environment state but on the a…
View article
Quantifying Registration Uncertainty With Sparse Bayesian Modelling Open
We investigate uncertainty quantification under a sparse Bayesian model of medical image registration. Bayesian modelling has proven powerful to automate the tuning of registration hyperparameters, such as the trade-off between the data an…
View article
Inference in Deep Gaussian Processes using Stochastic Gradient Hamiltonian Monte Carlo Open
Deep Gaussian Processes (DGPs) are hierarchical generalizations of Gaussian Processes that combine well calibrated uncertainty estimates with the high flexibility of multilayer models. One of the biggest challenges with these models is tha…
View article
Reducing Reparameterization Gradient Variance Open
Optimization with noisy gradients has become ubiquitous in statistics and machine learning. Reparameterization gradients, or gradient estimates computed via the "reparameterization trick," represent a class of noisy gradients often used in…
View article
Likelihood-free inference with emulator networks Open
Approximate Bayesian Computation (ABC) provides methods for Bayesian inference in simulation-based stochastic models which do not permit tractable likelihoods. We present a new ABC method which uses probabilistic neural emulator networks t…
View article
Density-Preserving Hierarchical EM Algorithm: Simplifying Gaussian Mixture Models for Approximate Inference Open
We propose an algorithm for simplifying a finite mixture model into a reduced mixture model with fewer mixture components. The reduced model is obtained by maximizing a variational lower bound of the expected log-likelihood of a set of vir…
View article
GAP: Generalizable Approximate Graph Partitioning Framework Open
Graph partitioning is the problem of dividing the nodes of a graph into balanced partitions while minimizing the edge cut across the partitions. Due to its combinatorial nature, many approximate solutions have been developed, including var…
View article
Neural Enhanced Belief Propagation on Factor Graphs Open
A graphical model is a structured representation of locally dependent random variables. A traditional method to reason over these random variables is to perform inference using belief propagation. When provided with the true data generatin…
View article
Being Bayesian, Even Just a Bit, Fixes Overconfidence in ReLU Networks Open
The point estimates of ReLU classification networks---arguably the most widely used neural network architecture---have been shown to yield arbitrarily high confidence far away from the training data. This architecture, in conjunction with …
View article
Structured Inference for Recurrent Hidden Semi-markov Model Open
Segmentation and labeling for high dimensional time series is an important yet challenging task in a number of applications, such as behavior understanding and medical diagnosis. Recent advances to model the nonlinear dynamics in such time…
View article
Incremental and Approximate Inference for Faster Occlusion-based Deep CNN Explanations Open
Deep Convolutional Neural Networks (CNNs) now match human accuracy in many image prediction tasks, resulting in a growing adoption in e-commerce, radiology, and other domains. Naturally, explaining CNN predictions is a key concern for many…
View article
DP-Forward: Fine-tuning and Inference on Language Models with Differential Privacy in Forward Pass Open
Differentially private stochastic gradient descent (DP-SGD) adds noise to gradients in back-propagation, safeguarding training data from privacy leakage, particularly membership inference. It fails to cover (inference-time) threats like em…