Intrinsic Motivation, Cumulative Learning, and Computational Reinforcement Learning

Abstract

Motivation refers to processes that influence the arousal, strength, and direction of behavior. Psychologists distinguish between extrinsic motivation, which means doing something because of some specific rewarding outcome, and intrinsic motivation, which refers to doing something because it is inherently enjoyable. Intrinsic motivation leads organisms to engage in exploration, play, and other behavior driven by curiosity in the absence of externally-supplied rewards. Intrinsically motivated learning has long been viewed as essential for the cumulative development of an animal's competence in interacting with the world. In this talk, I review some of the research on intrinsically motivated machine learning, which is not at all a new idea though it is receiving increased attention. I focus in particular on intrinsically motivated reinforcement learning (RL). A guiding principle is that the learning and behavior-generating processes in RL "don't care" if reward signals are intrinsic or extrinsic; the same processes can be used for both. But what is the difference between intrinsic and extrinsic rewards? I describe some recent computational experiments that may help us answer that question.