CISE Seminar: Vassilis Digalakis Jr., Assistant Professor of Operations & Technology Management, Boston University

ML Compass: Navigating Capability, Cost, and Compliance Trade-offs in AI Model Deploym

In this talk, I will discuss how organizations should select among competing AI models when user utility, deployment costs, and compliance requirements jointly matter. The talk relies on joint work with Ramayya Krishnan (CMU), Gonzalo Martin Fernandez (UPC), Agni Orfanoudaki (Oxford). The paper is available at https://www.arxiv.org/abs/2512.23487 Widely used capability leaderboards do not translate directly into deployment decisions, creating a capability -- deployment gap; to bridge it, we take a systems-level view in which model choice is tied to application outcomes, operating constraints, and a capability-cost frontier. We develop ML Compass, a framework that treats model selection as constrained optimization over this frontier. On the theory side, we characterize optimal model configurations under a parametric frontier and show a three-regime structure in optimal internal measures: some dimensions are pinned at compliance minima, some saturate at maximum levels, and the remainder take interior values governed by frontier curvature. We derive comparative statics that quantify how budget changes, regulatory tightening, and technological progress propagate across capability dimensions and costs. On the implementation side, we propose a pipeline that (i) extracts low-dimensional internal measures from heterogeneous model descriptors, (ii) estimates an empirical frontier from capability and cost data, (iii) learns a user- or task-specific utility function from interaction outcome data, and (iv) uses these components to target capability-cost profiles and recommend models. We validate ML Compass with two case studies: a general-purpose conversational setting using the PRISM Alignment dataset and a healthcare setting using a custom dataset we build using HealthBench. In both environments, our framework produces recommendations -- and deployment-aware leaderboards based on predicted deployment value under constraints -- that can differ materially from capability-only rankings, and clarifies how trade-offs between capability, cost, and safety shape optimal model choice.

Vassilis Digalakis Jr s an Assistant Professor of Operations & Technology Management (Questrom School of Business), also affiliated with the Division of Systems Engineering (College of Engineering) at Boston University. He holds a Ph.D. in Operations Research from MIT and a Diploma in Electrical and Computer Engineering from the Technical University of Crete, Greece. He works on trustworthy AI: how to design AI systems that behave reliably, transparently, and appropriately for their context of use. His research combines machine learning, optimization, and operations research to build such systems and to study how their deployment and governance shape adoption, downstream decisions, and real-world outcomes, particularly in healthcare and sustainability.

Faculty Host: Ayse Coskun

Student Host: Jiatong Guo