Building Safe and Trustworthy AI Systems
Artificial intelligence (AI) is everywhere, powering applications such as Spotify music suggestions, facial recognition from your smartphone or the ETA of your Uber. Neural networks are also being explored as controllers in a breadth of safety-critical systems, from piloting drones to detecting anomalies in nuclear power plants to maintaining first responder communication systems. At the same time, AI is vulnerable to cyber-attacks that can go undetected. AI is also increasingly complex, making it difficult to understand how the model decides. How can we trust the machine if we don’t fully understand it?
Professor Wenchao Li (ECE) leads the Dependable Computing Laboratory at Boston University where he and his team are addressing these challenges. The researchers have developed a combination of computational proof methods (a.k.a. formal methods) and machine learning techniques to make AI systems more trustworthy.
“Safety and security concerns are significant hurdles hindering the widespread adoption of AI and AI-enabled systems,” says Wenchao Li, Assistant Professor of Electrical and Computer Engineering (ECE) and Systems Engineering (SE) at Boston University. “Large-scale deployments of deep learning systems rely critically on their trustworthiness which, in turn, depends on the ability to assess and demonstrate the safety of such systems. There’s still a lack of guarantees on the reliability of these systems.”
An artificial neural network is a computational model, inspired by the brain, that is trained to learn, recognize patterns, and make decisions. Neural networks are vulnerable to adversarial attacks designed to trick the system, at the input, into making incorrect decisions, at the output. The classic example of this is a stop sign altered by a tiny perturbation designed to fool driverless cars to interpret it as a speed limit sign.
“The vulnerability of deep neural networks to adversarial examples has spurred the development of training methods for learning more robust models,” says Li . “There is also growing recognition in the field that models need to be certified as robust to adversarial examples.”
In their paper entitled Adversarial Training and Provable Robustness, Li and his students present a principled framework called AdvIBP that combines adversarial training and provable robustness verification for training certifiably robust neural networks. In this work, they also present a novel gradient descent method for two-objective optimization that uses moment estimates to address the issue of bias in stochastic multi-gradients. The researchers validated their method on a set of commonly used benchmarks and demonstrated that AdvIBP can learn provably robust neural networks that match or out-perform state-of-art techniques.
“This research has the potential to enable the efficient training of robust deep learning systems,” says Li. “It can help unlock deep learning applications that are currently not deployable due to safety, robustness or security concerns.”
Li’s research in AI and AI-enabled systems address the many multi-dimensional challenges to improving their safety and security, including novel methods and techniques in areas such as reachability analysis for neural-network controlled systems, neural trojan attacks and defenses, neural-network repair and reinforcement learning. With a focus on obtaining provable guarantees on systems, their work spans a breadth of applications, from electronic design automation, through multi-robot systems, to self-driving cars. Learn more about Li’s work here.
“Neural networks are hugely important,” says Li. “Neural networks have transformed the way we approach problems in many different applications. Making an impact on these problems has the potential to change people’s lives, not just from the technical aspect but on the actual adoption of these systems.”
Prior to joining BU, Li was a Computer Scientist in the Computer Science Laboratory at SRI International, Menlo Park. He received a B.S., M.S. and Ph.D. in Electrical Engineering and Computer Sciences and a B.A. in Economics from the University of California, Berkeley. His Ph.D. thesis on specification mining was awarded the ACM Outstanding Ph.D. Dissertation Award in Electronic Design Automation. He also received the Leon O. Chua Award at UC Berkeley for outstanding achievement in the area of nonlinear science.
His publications can be found here.
By Margaret Stanton, CISE Staff Writer