Ph.D. Research Talk: Sparse Variational Inference for Generalized Gaussian Process Models

Abstract

Gaussian processes (GP) provide an attractive machine learning model due to their non-parametric form, their flexibility to capture many types of observation data, and their generic inference procedures. Sparse GP inference algorithms address the cubic complexity of GPs by focusing on a small set of pseudo-examples. To date, such approaches have focused on the simple case of Gaussian likelihoods for the observations, or have couched sparse solutions within a latent variable GP model. In this talk, we describe a direct sparse solution for GPs under general likelihoods by providing a novel characterization of the gradients required for inference in terms of individual observation likelihood terms. In addition, we propose a simple new approach for optimizing the sparse variational approximation using a fixed point computation. We demonstrate experimentally that the fixed point operator acts as a contraction in many cases and therefore leads to fast convergence. An experimental evaluation for count regression, classification, and ordinal regression illustrates the generality and advantages of the new approach.