ECE/ME Seminar: Lukas Schmid

ECE/ME Seminar: Lukas Schmid

Title: Spatio-Temporal AI for Long-Term Human-Centric Robot Autonomy

Abstract: The ability to build an actionable representation of the environment of a robot is crucial for autonomy and prerequisite to a large variety of applications, ranging from home, service, and consumer robots to social, care, and medical robotics to industrial, agriculture and disaster response applications. Notably, a large part of the promise of autonomous robots depends on long-term operation in domains shared with humans and other agents. These environments are typically highly complex, semantically rich, and highly dynamic with agents frequently moving through and interacting with the scene.

This talk presents an autonomy pipeline combining perception, prediction, and planning to address these challenges. We first present methods to detect and represent complex semantics, short-term motion, and long-term changes for real-time robot perception in a unified framework called Khronos. We then show how Dynamic Scene Graphs (DSGs) can represent semantic symbols in a task-driven fashion and facilitate reasoning about the scene, such as the prediction of likely future outcomes based on the data the robot has already collected. Lastly, we show how robots as embodied agents can leverage these actionable scene representations and predictions to complete tasks such as actively gathering data that helps them improve their world models, perception, and action capabilities fully autonomously over time.

The presented methods are demonstrated on-board fully autonomous aerial, legged, and wheeled robots, run in real-time on mobile hardware, and are available as open-source software.

Bio: Lukas Schmid is a postdoctoral fellow at the MIT SPARK Lab led by Prof. Luca Carlone at the Massachusetts Institute of Technology (MIT). Before, he briefly was a postdoctoral researcher at the Autonomous Systems Lab (ASL) led by Prof. Roland Siegwart at ETH Zürich. He earned his PhD in 2022 from ETH Zurich advised by Prof. Roland Siegwart, where he also obtained his M.Sc. in Robotics, Systems, and Control (RSC) in 2019. During his PhD, he was a visiting researcher at the Microsoft Mixed Reality and AI Lab led by Prof. Marc Pollefeys.

His work has been recognized by several honors, including the RSS 2024 outstanding systems paper award, the ETH Medal for outstanding PhD Theses, the ETH Medal for outstanding Master Theses, the Willi Studer Prize for the best M.Sc. graduate of the year at ETH, the first place in the 2024 Hilti SLAM challenge, and a Swiss National Science Foundation postdoc fellowship.

His research focuses on active and passive perception of complex, dynamic, and changing environments for robotic interaction and augmented reality. This includes research on dense geometric and semantic scene representations, scene abstraction and understanding, as well as detection and prediction of moving and changing entities to enable continuous improvement of a robot's scene model and perception capabilities.