AIR Seminar, Fuxin Li, Professor, Oregon State University

Speaker: Fuxin Li, PhD, Professor in the School of Electrical Engineering and Computer Science at Oregon State University

Talk Title: "From Heatmaps to Structural and Counterfactual Explanations"

Abstract: This talk will focus on our endeavors in the past few years on explaining deep networks on images, where we believe we have shed significant light into how they make decisions. Realizing that an important missing piece for explaining neural networks is a reliable heatmap visualization tool, we developed I-GOS and iGOS++ which optimize with integrated gradients to avoid local optima in heatmap generations and improve performance in high-resolution heatmaps. Especially, iGOS++ was able to discover that deep classifiers trained on COVID-19 X-ray images wrongly focus on the characters printed on the image and could produce erroneous solutions. This shows the utility of explanation in "debugging" deep classifiers which have also recently been extended into explaining when and how do the image part matter in multimodal vision-language models.

During the development of those visualizations, we realize that for a significant number of images, the classifier has multiple different paths to reach a confident prediction. This leads to the development of structural attention graphs, an approach that utilizes beam search to locate multiple coarse heatmaps for a single image, and compactly visualizes a set of image masks by capturing how different combinations of image regions impact the confidence of a classifier. A user study shows significantly better capability of users to answer counterfactual questions when presented with SAG versus conventional heatmaps. We will also show our findings from running those explanation algorithms on recently popular transformer models, which indicate different decision-making behaviors among CNNs, global attention models and local attention models, which we characterize as disjunctive and compositional behaviors, respectively. Upon further examination, we found, to our surprise, that the choice of normalization layer is an important factor in different decision-making behaviors from networks. This work won a best student paper runner-up prize at CVPR.

Finally, as humans prefer visually appealing explanations that will literally “change” one class into another, we present results traversing the latent space of variational autoencoders and generative adversarial networks (GANs), generating high-quality counterfactual explanations that visually show how to change one image so that CNNs predict them as another category, without needing to re-train the autoencoders/GANs. When the classifier relies on wrong information to make classifications, the counterfactual explanations will illustrate the errors clearly. Counterfactual explanations also motivated us to look into long-tail data generation with diversity, where the tail class has significantly less data than the popular classes.

Speaker Bio: Fuxin Li has held research positions at Apple Inc., University of Bonn and Georgia Institute of Technology. He had obtained a Ph.D. degree in the Institute of Automation, Chinese Academy of Sciences in 2009. He has won an NSF CAREER award, an Amazon Research Award, CVPR 2024 Best Student Paper runner-up award, (co-)won the PASCAL VOC semantic segmentation challenges from 2009-2012, and led a team to the 4th place finish in the DAVIS Video Segmentation challenge 2017. He is a program chair of CVPR 2025. He has published more than 90 papers in computer vision, machine learning, as well as applications of machine learning and computer vision. His main research interests are point cloud deep networks, human understanding of deep learning, video object segmentation, multi-target tracking and uncertainty estimation in deep learning.

Faculty Host: Boqing Gong, Assistant Professor in Computer Science at Boston University

Back to Calendar