Dr Eric Schulz | Using structure to explore efficiently

Many types of intelligent behavior can be framed as a search problem, where an individual must explore a vast set of possible actions, while carefully balancing the exploration-exploitation dilemma. Under finite search horizons, optimal solutions are normally unobtainable. Yet humans and other animals regularly manage to solve these problems gracefully. How do they accomplish this? We propose an explanation based on two principles: generalization over features and uncertainty-guided exploration. Together these form a model that learns from past observations to generalize to similar options and eagerly seeks out uncertainty to gain more information about the search space. This model can be used to predict participants' search behavior in a complex multi-armed bandit task. Its parameter estimates can also be used to gain meaningful insights into developmental differences in generalization and directed exploration. Furthermore, we can use our model to describe customers' purchasing decisions in a large-scale data set of 1.6 million online food delivery purchases. Finally, I will end by describing ongoing work that puts this model to a test in a multi-armed bandit task with rats, in which we find similar principles influencing the animals' motor variability.

Poster