SE PhD Prospectus Defense of Vittorio Giammarino

TITLE: ON THE USE OF EXPERT DATA TO IMPROVE EFFICIENCY IN REINFORCEMENT LEARNING

ABSTRACT:This dissertation examines the integration of expert datasets to enhance the data efficiency of online Deep Reinforcement Learning (DRL) algorithms in the context of large state and action space problems. The focus is on effectively integrating real-world data, including data from biological systems, to accelerate the learning process within the online DRL pipeline. The motivation for this work is twofold: first, the internet provides access to a vast amount of data, such as videos, that demonstrate various tasks of interest, but are not necessarily designed for use in the DRL framework. Leveraging this data to enhance DRL algorithms presents an exciting and challenging opportunity. Second, biological systems exhibit numerous inductive biases in their behavior that enable them to be highly efficient and adaptable learners. Incorporating these mechanisms for efficient learning remains an open question in DRL, and we consider the use of human and animal data as a possible solution to this problem.

Throughout this dissertation, we address important questions such as how prior knowledge can be distilled into RL agents, the benefits of leveraging offline datasets for online RL, and the algorithmic challenges involved in doing so. We present three original works that investigate the use of animal videos to enhance RL learning performance, develop a framework to learn bio-inspired foraging policies using human data, and propose an online algorithm for performing hierarchical imitation learning in the options framework.

Our research demonstrates the effectiveness of utilizing offline datasets to improve the efficiency and performance of online DRL algorithms, providing valuable insights into how we can accelerate the learning process for complex tasks.

COMMITTEE: AdvisorIoannis Paschalidis, SE, ECE; Eshed Ohn-Bar, ECE; Wenchao Li, SE, ECE; Roberto Tron, SE, ME

When 11:00 am to 1:00 pm on Friday, April 28, 2023
Location 8 Saint Mary's Street, RM 442