A Framework for Curriculum Schema Transfer from Low-Fidelity to High-Fidelity Environments

November 18, 2022
3:00 PM ET
Cummings 160
Speaker: Yash Shukla
Host: Jivko Sinapov

Abstract

Quals talk:

Despite recent advances in Reinforcement Learning (RL), many problems, especially real-world tasks, remain prohibitively expensive to learn. To address this issue, several lines of research have explored how tasks, or data samples themselves, can be sequenced into a curriculum to learn a problem that may otherwise be too difficult to learn from scratch. However, generating and optimizing a curriculum in a realistic scenario still requires extensive interactions with the environment. To address this challenge, I formulate the curriculum transfer problem, in which the schema of a curriculum optimized in a simpler, easy-to-solve environment (e.g., a gridworld) is transferred to a complex, realistic scenario (e.g., a physics-based robotics simulation or the real world). In my presentation, I will present “ACuTE”,Automatic Curriculum Transfer from Simple to Complex Environments, a novel framework to solve this problem, and evaluate our proposed method by comparing it to other baseline approaches (e.g., domain adaptation) designed to speed up learning. My approach produces improved jumpstart and time-to-threshold performance even when adding task elements that further increase the difficulty of the realistic scenario. Finally, I will also demonstrate that our approach is independent of the learning algorithm used for curriculum generation, and is Sim2Real transferable to a real-world scenario using a physical robot. This work was recently published as a full paper at AAMAS 2022.

Research area: Reinforcement Learning; Curriculum Learning; Robot Learning.

Please join meeting in Cummings 160

Zoom is not available for this event; disregard dial-in passcode included in email.