MechE PhD Prospectus Defense: Yancheng Zhu

TITLE: Study of Optimal Control Policy for Data Harvesting in Wireless Sensor Networks

ABSTRACT: Over the last decade, the technologies available for collecting and analyzing large volumes of data have exploded. With this wealth of technologies comes the challenge of how to extract such volumes of data from sensor nodes distributed over large, often remote, geographical regions. Data harvesting is the problem of extracting measurements from the remote nodes of a Wireless Sensor Network (WSN) using mobile elements such as ground vehicles or drones. In this prospectus, we set the stage for our research by describing our efforts to obtain the optimal control policy for data harvesting problem in 1-D space using a collection of mobile agents. We use a Hamiltonian analysis to show that the optimal control can be described using a parameterized policy and then develop a gradient descent scheme using infinitesimal perturbation analysis (IPA) to calculate the gradients of the cost function with respect to the control parameters. Next, to avoid collisions between agents, we move on to a Control Lyapunov Function-Control Barrier Function (CLF-CBF) technique to ensure the agents closely track the desired optimal trajectory to complete their mission while avoiding any collisions. After that, we translate the problem to a Markov Decision Process in discrete time, and then apply reinforcement learning to find high performing solutions using double deep Q learning. We demonstrate our approaches through several simulations and explore the effect of the learning method in the 2-D setting. We then describe our proposed future work. We plan to extend our optimal control approach to more complicated environments and to consider the impact of mobility of the sensor nodes and of stochastic variation of the data transmission. We also intend to study the impact of optimal management of the agent-sensor communication channel, accounting for power limitations of the sensor nodes and the constraints this imposes on data harvesting. Lastly, as the research of WSNs has revealed that the signal transmitting power is highly affected by obstacles, we tend to apply learning approaches to explore the unknown environment and generate the optimal control policy for the UAVs based on the signal intensity map we have learned.

COMMITTEE: ADVISOR/CHAIR Professor Sean Andersson, ME/SE; Professor Christos Cassandras, ECE/SE; Professor Roberto Tron, ME/SE; Professor Alyssa Pierson, ME