Black Box Variational Inference
Probabilistic generative models are robust to noise, uncover unseen patterns, and make predictions about the future. These models have been used successfully to solve problems in neuroscience, astrophysics, genetics, and medicine. The main computational challenge is computing the hidden structure given the data --- posterior inference. For most models of interest, computing the posterior distribution requires approximations like variational inference. Classically, variational inference was feasible to deploy in only a small fraction of models. We develop black box variational inference. Black box variational inference is a variational inference algorithm that is easy to deploy on a broad class of models and has already found use in neuroscience and healthcare. Finally, the ideas around black box variational inference also facilitate new kinds of variational methods. We develop hierarchical variational models. Hierarchical variational models improve the approximation quality of variational inference by building higher-fidelity approximations from coarser ones. Black box variational inference opens the doors to new models and better posterior approximations.
Rajesh Ranganath is a staff associate at Columbia University's Department of Statistics and a research affiliate at MIT's Institute for Medical Engineering and Science. He will be an assistant professor at the Courant Institute of Mathematical Sciences at NYU starting January 2018. His research interests include approximate inference, model checking, Bayesian nonparametrics, and machine learning for healthcare. Rajesh completed his PhD at Princeton with David Blei. Before starting his PhD, Rajesh worked as a software engineer for AMA Capital Management. He obtained his BS and MS from Stanford University with Andrew Ng and Dan Jurafsky. Rajesh has won several awards and fellowships including the NDSEG graduate fellowship and the Porter Ogden Jacobus Fellowship, given to the top four doctoral students at Princeton University.