Towards More Automated Machine Learning: High-Dimensional Bayesian Optimization and Probabilistic Neural Architecture Search
State-of-the-art machine learning/AI systems crucially depend on identifying the correct model structure, model hyperparameters and data processing pipeline. If any of these components is misspecified or badly tuned, the resulting system may not work at all. ML practitioners often resort to manual tuning or brute force approaches, but the optimization space can be too complex and high-dimensional to explore properly with these methods. When automated systems are used, the high costs of running a single experiment (e.g. training a deep neural network) and the high sample complexity (i.e. large number of experiments required) make naïve approaches impractical. In this talk, we will discuss two problems in automated machine learning and show how probabilistic machine learning models can guide (automated) experimental decisions and how meta-learning can be used to transfer knowledge across related datasets or problems, hence reducing the sample complexity.