ECE MS Thesis Defense: Xulun Huang

ECE MS Thesis Defense: Xulun Huang

Title: Low-Power, Low-Complexity Image Classification with Optical Sensors

Presenter: Xulun Huang

Advisor: Professor Janusz Konrad

Committee: Professor Roberto Paiella and Professor Brian Kulis

Abstract: Convolutional Neural Networks (CNNs) have become a very effective tool in image estimation and inference (e.g., noise reduction, segmentation, object recognition). However, CNNs are difficult to deploy in resource-constrained scenarios, such as edge devices, IoT sensors, mobile embedded systems, because of high computational requirements and large memory footprint. Clearly, there is a growing need to investigate lightweight neural-network architectures designed to provide high performance at significantly reduced computational cost, memory usage, and power consumption.

In this thesis, we investigate a low-power, low-complexity hybrid optical-digital neural network that leverages a novel metasurface sensor recently developed in Professor Paiella’s lab. Unlike typical image sensors, this sensor outputs an edge-like map of the scene akin to the output of the first convolutional layer of a CNN. We simulate this physical sensor in software and combine it with just a few digital layers to assure low computational load and power consumption. Since different pixels of the sensor capture different edge orientations, we organize the sensor array into groups of 2-by-2 or 3-by-3 pixels capturing either 4 or 9 edge orientations. This leads to either 4-channel or 9-channel convolutional layer simulation. We jointly optimize the optical parameters of this layer and digital parameters of the remaining layers for image classification of low-resolution images. Our best-performing designs approach classification performance of equivalent fully-digital network within 2%, while reducing computational complexity (and power consumption) by a factor of 4.