ECE Seminar: Beyazit Yalcinkaya

ECE Seminar: Beyazit Yalcinkaya

Title: Formal Specification Embeddings for Task-Conditioned Reinforcement Learning

Abstract: Recent success of foundation models has popularized pretrained language and image embeddings as instruction modalities for autonomous agents. However, ensuring the correctness of learned policies remains a major challenge due to the inherent ambiguity of natural language and demonstrations. In this talk, I will present our work on formal specification embeddings and their use in task-conditioned policy learning. The key idea is to pretrain latent representations of formal specifications and condition downstream policies on these frozen embeddings, thereby decoupling representation learning from control learning. In this setting, theoretical guarantees on the learned embeddings enable probably approximately correct task-conditioned policy learning for long-horizon objectives. Beyond their theoretical advantages, pretrained specification embeddings improve sample efficiency, rendering learning-based control practical in settings that would otherwise be infeasible to train. I will present applications of this framework in both single-agent and multi-agent settings, demonstrating the effectiveness of formal specification embeddings for learning-based control. I will also briefly discuss a comparative study of natural-language instruction embeddings and formal-specification embeddings for downstream task-conditioned policy learning. Results suggest that formal specification embeddings enable efficient task-conditioned policy learning with correctness guarantees and that they provide a complementary instruction modality to language and demonstrations.

Bio: Beyazit Yalcinkaya is a PhD candidate in Computer Science at University of California, Berkeley, advised by Sanjit A. Seshia. He received his B.S. in Computer Engineering from Middle East Technical University, Ankara, Turkiye. Prior to joining Berkeley, he conducted research at the Max Planck Institute for Software Systems, Kaiserslautern, Germany, and at EPFL, Lausanne, Switzerland. His research combines formal methods and reinforcement learning, with an emphasis on specification-guided multi-task and multi-agent policy learning in model-free settings.