Chen Sun, Assistant Professor, Brown University

Talk Title : Rethinking Video Representation Learning for Perception and Reasoning in the Era of LLMs

Abstract: What makes good representations for video understanding, such as inferring the intents of human actors, answering video-grounded questions based on (physical) commonsense, or solving reasoning tasks that require compositional generalization? In this talk, I will present our recent attempts to answer this research question from three perspectives: First, we investigate if an end-to-end trained neural network (such as a VLM) is capable of solving complex visual reasoning tasks that traditionally tackled by neuro-symbolic methods; Second, we explore if structured representations (such as objects and attributes, abstract representation of the environments, or visual question decomposition) would emerge automatically with large-scale pre-training, without manual definitions of “concepts” or “symbols”; Finally, we study what makes good representations for video understanding tasks, especially on bridging perception and reasoning for long-form video understanding.

Bio: Chen Sun is an Assistant Professor of Computer Science at Brown University and a researcher at Google DeepMind, studying computer vision and machine learning. His lab’s research has received a Richard B. Salomon Faculty Research Award, Samsung’s Global Research Outreach Award, and appeared in the best paper finalist of CVPR 2019. Chen received his Ph.D. from the University of Southern California in 2016, and bachelor degree from Tsinghua University in 2011.

Back to Calendar