Learning Models that See Through Walls and Sense Sleep and Vital Signs

BU CS Distinguished Lecture Series - Professor Dina Katabi, MIT

Abstract: Tracking people through walls and occlusions is an important task with applications in activity recognition, gaming, home security and health monitoring. In this talk, I introduce neural network models that infer the human Pose through obstacles, i.e., they infer the skeletal representations of the joints on the arms and legs, and the keypoints on the torso and head. Our models leverage that wireless signals in the WiFi frequencies traverse walls and reflect off the human body. They parse such radio signals to estimate human poses. Since no signal annotations exist for such task, we use state-of-the-art computer vision models to provide cross-modal supervision. Interestingly, while our models are trained using a camera, they can estimate human poses through walls and occlusions. I also explain how to extend these models to infer a person's breathing, heart rate, and sleep stages from the surrounding wireless signals, without any sensor on the person's body. Finally, I discuss how our technology can be used for in-home patient monitoring to deliver better care to chronic disease patients.

Bio: Dina Katabi is the Andrew & Erna Viterbi Professor of Electrical Engineering and Computer Science at MIT. She is also the director of the MIT’s Center for Wireless Networks and Mobile Computing, a member of the National Academy of Engineering, and a recipient of the MacArthur Genius Award. Professor Katabi received her PhD and MS from MIT in 2003 and 1999, and her Bachelor of Science from Damascus University in 1995. Katabi's research focuses on innovative mobile and wireless technologies with application to digital health. Her research has been recognized with ACM Prize in Computing, the ACM Grace Murray Hopper Award, the SIGCOMM test of Time Award, the Faculty Research Innovation Fellowship, a Sloan Fellowship, the NBX Career Development chair, and the NSF CAREER award. Her students received the ACM Best Doctoral Dissertation Award in Computer Science and Engineering twice. Further, her work was recognized by the IEEE William R. Bennett prize, three ACM SIGCOMM Best Paper awards, an NSDI Best Paper award, and a TR10 award. Several start-ups have been spun out of Katabi's lab such as PiCharging and Emerald.