PhD Defense: Flight Controller Synthesis via Deep Reinforcement Learning
Wil Koch, PhD Candidate, Boston University, Computer Science, CAS
When:
Friday, August 9, 2019
Event Start Time: 2:00 pm
Event End Time: 3:30 pm (Refreshments to follow)
Where:
Hariri Institute for Computing, 111 Cummington Mall, Seminar Room, MCS 180
Abstract: Traditional control methods are inadequate in many deployment settings involving autonomous control of Cyber-Physical Systems (CPS). In such settings, CPS controllers must operate and respond to unpredictable interactions, conditions, or failure modes. Dealing with such unpredictability requires the use of executive and cognitive control functions that allow for planning and reasoning. Motivated by the sport of drone racing, this dissertation addresses these concerns for state-of-the-art in-flight control by investigating the use of deep artificial neural networks to bring essential elements of higher-level cognition to bear on the design, implementation, deployment, and evaluation of low level (attitude) flight controllers.
First, this thesis presents feasibility analyses and results which confirm that neural networks, trained via reinforcement learning, are more accurate than traditional control methods used by commercial uncrewed aerial vehicles (UAVs) for attitude control. Second, armed with these results, this thesis reports on the development and release of an open-source, full solution stack for building neuro-flight controllers. This stack consists of a tuning framework for implementing training environments (GymFC) and firmware for the world’s first neural-network-supported flight controller (Neuroflight). GymFC’s novel approach fuses the digital twinning paradigm with flight control training to provide seamless transfer to hardware. Third, to transfer models synthesized by GymFC to hardware, this thesis reports on the toolchain that has been released for compiling neural networks into Neuroflight, which can be flashed to off-the-shelf microcontrollers. This toolchain includes detailed procedures for constructing a multi-copter digital twin to allow the research and development community to synthesize flight controllers unique to their aircraft. Finally, this thesis examines alternative reward system functions as well as changes to the software environment to bridge the gap between simulation and real-world deployment environments.
The design, evaluation, and experimental work summarized in this thesis demonstrates that deep-reinforcement-learning can be leveraged for the design and implementation of neural-network controllers capable not only of maintaining stable flight but of executing aerobatic maneuvers in real-world settings. As such, this work provides a foundation for developing the next generation of flight control systems.
Thesis Committee:
- Azer Bestavros, Professor of Computer Science, CAS and Director of the Hariri Institute for Computing (Major Advisor and 1st Reader)
- Renato Mancuso, Assistant Professor, Computer Science, CAS (2nd Reader)
- Richard West, Professor, Computer Science, CAS (3rd Reader)
- Abraham Matta, Professor & Associate Chair, Computer Science, CAS (Committee Chair)
Bio: Wil Koch received his BS in computer engineering from University of Rhode Island where he developed interests in machine learning and cyber security. After graduation he worked as a software engineer for six years on the Sikorsky CH-53K King Stallion helicopter project before returning back to academia to receive his MS in machine learning from Stevens Institute of Technology. To focus on his growing interest in cyber security he came to Boston University for his PhD where he has been advised by Azer Bestavros. He has published a number of papers in a wide area of cyber security including static and dynamic malware analysis, vulnerability analysis, cyber defense and attacks and mobile security. In 2017, Wil was introduced to first-person-view (FPV) drone racing and was immediately drawn to the sport for his love of flight, racing and engineering. This passion drove a shift in his research. He began working with Renato Mancuso, Richard West, and Azer Bestavros to fuse together machine learning and flight control leading to the development of the worlds first neural network flight control firmware, Neuroflight. As a leader and someone who encourages continued learning Wil has co-founded CT Hackerspace, Connecticut’s oldest and largest hackerspace and founded Boston Drone Racing, an FPV drone racing and hacking club.