Symbolic Bayesian Inference by Lazy Partial Evaluation

Chung-Chieh Shan and Norman Ramsey

Bayesian inference, of posterior knowledge based on prior knowledge and observed evidence, is typically implemented by applying Bayes's theorem, solving an equation in which the posterior multiplied by the probability of an observation equals a joint probability. But when we observe a value of a continuous variable, the observation usually has probability zero, and Bayes's theorem says only that zero times the unknown is zero. To infer a posterior distribution from a zero-probability observation, we turn to the statistical technique of disintegration. The classic formulation of disintegration tells us only what constitutes a posterior distribution, not how to compute it. But by representing all distributions and observations as terms of our probabilistic language, core Hakaru, we have developed the first constructive method of computing disintegrations, solving the problem of drawing inferences from zero-probability observations. Our method uses a lazy partial evaluator to transform terms of core Hakaru, and we argue its correctness by a semantics of core Hakaru in which monadic terms denote measures. The method, which has been implemented in a larger system, is useful not only on its own but also in composition with sampling and other inference methods commonly used in machine learning.

Full paper

The paper is available as US Letter PDF (291K)