Inference in Probabilistic Programs

Abstract

Probabilistic models in machine learning have traditionally co-evolved with tailored methods for posterior inference and parameter learning. Much in the way that a programmer writing assembly code must have a mental model of both the desired program function and the underlying processor architecture, a machine learning practitioner must have a low-level understanding of both the structure of the probabilistic model and the corresponding inference implementation.

Probabilistic programming systems aim to accelerate iterative development of machine learning approaches by introducing an abstraction boundary: models are defined using a domain-specific language, and a back end implements generic inference methods for such programs. The aim of this research endeavor is to do for the domains of data science and artificial intelligence what compiler technologies have done for software development: enable practitioners to reason about their models at a higher level of abstraction.

In this talk I will discuss inference strategies employed in Anglican, a probabilistic programing system closely integrated with the language Clojure. Anglican has pioneered inference techniques based on sequential Monte Carlo that apply to programs written in general-purpose languages that support recursion, higher-order functions, and black box deterministic simulation steps. In addition to inference strategies, I will discuss applications, and opportunities for future research in this emerging domain.

Bio: Jan-Willem is post-doc in Machine Learning at the Department of Engineering Science at Oxford. He works primarily on the Anglican probabilistic programming system, which he created with Frank Wood and David Tolpin. His broader research agenda is to understand how programs may be used to define structured and composable models for machine learning and artificial intelligence. To facilitate this agenda, he also works on inference techniques for probabilistic programs.

Prior to joining Oxford, Jan-Willem has worked on machine learning methods for biophysics with Chris Wiggins and Ruben Gonzalez at Columbia University. He did his PhD research in biological fluid mechanics with Ray Goldstein (Cambridge) and Wim van Saarloos (Leiden).