Benjamin Letham
YOU?
Author Swipe
View article: Experimenting, Fast and Slow: Bayesian Optimization of Long-term Outcomes with Online Experiments
Experimenting, Fast and Slow: Bayesian Optimization of Long-term Outcomes with Online Experiments Open
Online experiments in internet systems, also known as A/B tests, are used for a wide range of system tuning problems, such as optimizing recommender system ranking policies and learning adaptive streaming controllers. Decision-makers gener…
View article: Mixed Likelihood Variational Gaussian Processes
Mixed Likelihood Variational Gaussian Processes Open
Gaussian processes (GPs) are powerful models for human-in-the-loop experiments due to their flexibility and well-calibrated uncertainty. However, GPs modeling human responses typically ignore auxiliary information, including a priori domai…
View article: Onset timing of letter processing in auditory and visual sensory cortices
Onset timing of letter processing in auditory and visual sensory cortices Open
Here, we report onset latencies for multisensory processing of letters in the primary auditory and visual sensory cortices. Healthy adults were presented with 300-ms visual and/or auditory letters (uppercase Roman alphabet and the correspo…
View article: Active Learning for Derivative-Based Global Sensitivity Analysis with Gaussian Processes
Active Learning for Derivative-Based Global Sensitivity Analysis with Gaussian Processes Open
We consider the problem of active learning for global sensitivity analysis of expensive black-box functions. Our aim is to efficiently learn the importance of different input variables, e.g., in vehicle safety experimentation, we study the…
View article: Perceptual Requirements for World-Locked Rendering in AR and VR
Perceptual Requirements for World-Locked Rendering in AR and VR Open
Stereoscopic, head-tracked display systems can show users realistic, world-locked virtual objects and environments. However, discrepancies between the rendering pipeline and physical viewing conditions can lead to perceived instability in …
View article: Poster Session I: Perspective-correct rendering for active observers
Poster Session I: Perspective-correct rendering for active observers Open
Stereoscopic, head-tracked display systems can show users realistic, world-locked virtual objects and environments (i.e., rendering perspective-correct binocular images with accurate motion parallax). However, discrepancies between the ren…
View article: A Semi-parametric Model for Decision Making in High-Dimensional Sensory Discrimination Tasks
A Semi-parametric Model for Decision Making in High-Dimensional Sensory Discrimination Tasks Open
Psychometric functions typically characterize binary sensory decisions along a single stimulus dimension. However, real-life sensory tasks vary along a greater variety of dimensions (e.g. color, contrast and luminance for visual stimuli). …
View article: Response Time Improves Choice Prediction and Function Estimation for Gaussian Process Models of Perception and Preferences
Response Time Improves Choice Prediction and Function Estimation for Gaussian Process Models of Perception and Preferences Open
Models for human choice prediction in preference learning and psychophysics often consider only binary response data, requiring many samples to accurately learn preferences or perceptual detection thresholds. The response time (RT) to make…
View article: Perceptual Requirements for World-Locked Rendering in AR and VR
Perceptual Requirements for World-Locked Rendering in AR and VR Open
Stereoscopic, head-tracked display systems can show users realistic, world-locked virtual objects and environments. However, discrepancies between the rendering pipeline and physical viewing conditions can lead to perceived instability in …
View article: A Semi-Parametric Model for Decision Making in High-Dimensional Sensory Discrimination Tasks
A Semi-Parametric Model for Decision Making in High-Dimensional Sensory Discrimination Tasks Open
Psychometric functions typically characterize binary sensory decisions along a single stimulus dimension. However, real-life sensory tasks vary along a greater variety of dimensions (e.g. color, contrast and luminance for visual stimuli). …
View article: Look-Ahead Acquisition Functions for Bernoulli Level Set Estimation
Look-Ahead Acquisition Functions for Bernoulli Level Set Estimation Open
Level set estimation (LSE) is the problem of identifying regions where an unknown function takes values above or below a specified threshold. Active sampling strategies for efficient LSE have primarily been studied in continuous-valued fun…
View article: Sparse Bayesian Optimization
Sparse Bayesian Optimization Open
Bayesian optimization (BO) is a powerful approach to sample-efficient optimization of black-box objective functions. However, the application of BO to areas such as recommendation systems often requires taking the interpretability and simp…
View article: Adaptive Nonparametric Psychophysics
Adaptive Nonparametric Psychophysics Open
We introduce a new set of models and adaptive psychometric testing methods for multidimensional psychophysics. In contrast to traditional adaptive staircase methods like PEST and QUEST, the method is multi-dimensional and does not require …
View article: Re-Examining Linear Embeddings for High-Dimensional Bayesian Optimization
Re-Examining Linear Embeddings for High-Dimensional Bayesian Optimization Open
Bayesian optimization (BO) is a popular approach to optimize expensive-to-evaluate black-box functions. A significant challenge in BO is to scale to high-dimensional parameter spaces while retaining sample efficiency. A solution considered…
View article: Re-Examining Linear Embeddings for High-Dimensional Bayesian Optimization
Re-Examining Linear Embeddings for High-Dimensional Bayesian Optimization Open
Bayesian optimization (BO) is a popular approach to optimize expensive-to-evaluate black-box functions. A significant challenge in BO is to scale to high-dimensional parameter spaces while retaining sample efficiency. A solution considered…
View article: BoTorch: A Framework for Efficient Monte-Carlo Bayesian Optimization
BoTorch: A Framework for Efficient Monte-Carlo Bayesian Optimization Open
Bayesian optimization provides sample-efficient global optimization for a broad range of applications, including automatic machine learning, engineering, physics, and experimental design. We introduce BoTorch, a modern programming framewor…
View article: Bayesian Optimization for Policy Search via Online-Offline Experimentation
Bayesian Optimization for Policy Search via Online-Offline Experimentation Open
Online field experiments are the gold-standard way of evaluating changes to real-world interactive machine learning systems. Yet our ability to explore complex, multi-dimensional policy spaces - such as those found in recommendation and ra…
View article: Constrained Bayesian Optimization with Noisy Experiments
Constrained Bayesian Optimization with Noisy Experiments Open
Randomized experiments are the gold standard for evaluating the effects of changes to real-world systems. Data in these tests may be difficult to collect and outcomes may have high variance, resulting in potentially large measurement error…
View article: Scalable Meta-Learning for Bayesian Optimization
Scalable Meta-Learning for Bayesian Optimization Open
Bayesian optimization has become a standard technique for hyperparameter optimization, including data-intensive models such as deep neural networks that may take days or weeks to train. We consider the setting where previous optimization r…
View article: Practical Transfer Learning for Bayesian Optimization
Practical Transfer Learning for Bayesian Optimization Open
When hyperparameter optimization of a machine learning algorithm is repeated for multiple datasets it is possible to transfer knowledge to an optimization run on a new dataset. We develop a new hyperparameter-free ensemble model for Bayesi…
View article: Forecasting at Scale
Forecasting at Scale Open
Forecasting is a common data science task that helps organizations with capacity planning, goal setting, and anomaly detection. Despite its importance, there are serious challenges associated with producing reliable and high quality foreca…
View article: Erratum: “Prediction uncertainty and optimal experimental design for learning dynamical systems” [Chaos <b>26</b>, 063110 (2016)]
Erratum: “Prediction uncertainty and optimal experimental design for learning dynamical systems” [Chaos <b>26</b>, 063110 (2016)] Open
First Page
View article: A Computational Model of Inhibition of HIV-1 by Interferon-Alpha
A Computational Model of Inhibition of HIV-1 by Interferon-Alpha Open
Type 1 interferons such as interferon-alpha (IFNα) inhibit replication of Human immunodeficiency virus (HIV-1) by upregulating the expression of genes that interfere with specific steps in the viral life cycle. This pathway thus represents…
View article: A computational model of inhibition of HIV-1 by interferon-alpha
A computational model of inhibition of HIV-1 by interferon-alpha Open
Type 1 interferons such as interferon-alpha (IFN α ) inhibit replication of Human immunodeficiency virus (HIV-1) by upregulating the expression of genes that interfere with specific steps in the viral life cycle. This pathway thus represen…
View article: Interpretable classifiers using rules and Bayesian analysis: Building a better stroke prediction model
Interpretable classifiers using rules and Bayesian analysis: Building a better stroke prediction model Open
We aim to produce predictive models that are not only accurate, but are also\ninterpretable to human experts. Our models are decision lists, which consist of\na series of if...then... statements (e.g., if high blood pressure, then stroke)\…
View article: Bayesian Inference of Arrival Rate and Substitution Behavior from Sales\n Transaction Data with Stockouts
Bayesian Inference of Arrival Rate and Substitution Behavior from Sales\n Transaction Data with Stockouts Open
When an item goes out of stock, sales transaction data no longer reflect the\noriginal customer demand, since some customers leave with no purchase while\nothers substitute alternative products for the one that was out of stock. Here\nwe d…
View article: Bayesian Inference of Arrival Rate and Substitution Behavior from Sales Transaction Data with Stockouts
Bayesian Inference of Arrival Rate and Substitution Behavior from Sales Transaction Data with Stockouts Open
When an item goes out of stock, sales transaction data no longer reflect the original customer demand, since some customers leave with no purchase while others substitute alternative products for the one that was out of stock. Here we deve…
View article: Statistical learning for decision making : interpretability, uncertainty, and inference
Statistical learning for decision making : interpretability, uncertainty, and inference Open
Thesis: Ph. D., Massachusetts Institute of Technology, Sloan School of Management, Operations Research Center, 2015.