ECE PhD Thesis Defense: Hoang Tran

ECE PhD Thesis Defense: Hoang Tran

Title: Towards Private And Efficient Optimization Methods For Machine Learning

Presenter: Hoang Tran

Advisor: Professor Ashok Cutkosky

Chair: Professor David Castañon

Committee: Professor Ashok Cutkosky, Professor Brian Kulis, Professor Kayhan Batmanghelich, Professor Alex Olshevsky

Google Scholar Link: https://scholar.google.com/citations?user=IdSgJnEAAAAJ&hl=en

Abstract: In the last couple of decades, we have seen tremendous progress in Machine Learning (ML) both in the scale and effectiveness of ML models. ML models are now used in every aspect of our life, from healthcare and finance to personalized recommendations and autonomous systems. As these models become more sophisticated, they are required to handle more data than ever. This has introduced a huge need to design ML algorithms that have good utility guarantees while ensuring privacy for users. This is where differentially private (DP) machine learning plays a critical role. Differential privacy provides a mathematically rigorous framework to ensure that individual data points cannot be reverse-engineered from the model’s outputs, even by someone with access to large amounts of auxiliary information.

This thesis introduces novel differentially private (DP) algorithms for various settings, including Non-convex Empirical Risk Minimization (ERM), Stochastic Convex Optimization (SCO), and Stochastic Linear Bandits. We begin by exploring the general Non-convex Optimization landscape, which is central to modern ML, and discuss the challenges involved in making non-private algorithms differentially private. Following this, we introduce the binary tree mechanism, demonstrating how it can be leveraged to develop new private algorithms that achieve state-of-the-art (SOTA) utility guarantees in both ERM and SCO settings. These results are made possible by exploiting the sensitivity-reducing properties of the tree mechanism. Finally, we present a new private variant of the classic LinUCB algorithm, which achieves asymptotically optimal regret bounds under specific settings for stochastic linear bandits.