Towards More Automated Machine Learning: High-Dimensional Bayesian Optimization and Probabilistic Neural Architecture Search

Abstract

State-of-the-art machine learning/AI systems crucially depend on identifying the correct model structure, model hyperparameters and data processing pipeline. If any of these components is misspecified or badly tuned, the resulting system may not work at all. ML practitioners often resort to manual tuning or brute force approaches, but the optimization space can be too complex and high-dimensional to explore properly with these methods. When automated systems are used, the high costs of running a single experiment (e.g. training a deep neural network) and the high sample complexity (i.e. large number of experiments required) make naïve approaches impractical. In this talk, we will discuss two problems in automated machine learning and show how probabilistic machine learning models can guide (automated) experimental decisions and how meta-learning can be used to transfer knowledge across related datasets or problems, hence reducing the sample complexity.

Bio

Nicolo Fusi is a Principal Research Manager at Microsoft Research, where he leads the automated machine learning research team. Beyond AutoML, his research interests include Gaussian processes, Bayesian nonparametrics and scalable inference methods. In computational biology, he has worked on statistical methods to perform genome-wide association studies and predictive models for CRISPR/Cas9 gene editing. Nicolo received his Ph.D. in Computer Science from the University of Sheffield working with Neil Lawrence. He received his B.Sc. and M.Sc. in computer science from the University of Milan.