UCL Logo
Ozgur Simsek

Center for Adaptive Behavior and Cognition, Max Planck Institute for Human Development, Germany


Wednesday 30 September 2009


 Seminar Room B10 (Basement)

Alexandra House, 17 Queen Square, London, WC1N 3AR


Behavioral building blocks for autonomous agents

 The broad problem I will address in this talk is design of autonomous agents that can efficiently learn how to achieve desired behaviors in large, complex environments. I will focus on one essential design component: the ability to form new behavioral units from existing ones. For example, a robot that routinely manipulates objects would benefit from forming a grasping behavior, using its lower-level sensory and motor actions, and using the grasping behavior as a (behavioral) unit in its decision making. Similarly, a tic-tac-toe player would benefit from acquiring the behavior to set up a fork on the board.

In this talk, I will first address the question of what makes a useful behavior. I will define a class of behaviors using a graphical representation of the agent’s interaction with its environment and a measure of centrality on graphs called betweenness. I will show that, in a diverse set of domains, these behaviors are consistent with common sense, are similar to skills that people handcraft for these domains, and improve learning performance. I will then propose methods that artificial agents can use to identify and acquire such behaviors autonomously.


hiererachical reinforcement learning, intrinsic motivation, skill learning