CAREER Award for AI Intermediary
by A.J. Kleber
Clinical medicine is arguably the area where AI-based tools and methodology show the most promise, but also the most potential for significant repercussions where the technology falls short. Professor Kayhan Batmanghelich’s new project, supported by a prestigious Faculty Early Career Development Program (CAREER) Award from the National Science Foundation (NSF), will endeavor to address some of these critical performance issues by using a new AI model to liaise between human users and existing medical models … rather than attempting to start from scratch.
AI diagnostic tools and predictive models, trained on medical images like CT scans, have the capacity for high accuracy, but also for misinterpretation and uneven, unreliable results; not a desirable risk when it comes to identifying and treating debilitating or deadly diseases! An AI agent is designed to recognize and replicate patterns; that doesn’t mean it will always learn the right patterns within a particular context.
It doesn’t help that so many AI models operate as “black boxes,” making it nearly impossible to evaluate their decision-making processes or identify how an error occurred. Another challenge is that the developers who design the technology, and the clinicians who use it, have wildly different areas of expertise and technical literacy.
Professor Batmanghelich, a machine learning researcher with considerable experience working with medical imagery, proposes a novel approach to these challenges: designing a novel AI system to identify and interpret the errors in the “black box” medical AI. His project, Making Domain-Specific AI Models Steerable by Leveraging Foundational Models, centers on the development of a new generation of Vision-Language Models (VLLMs), AI systems that combine image analysis with language-based reasoning. These models will be trained with improved “anatomical awareness”. These VLLMs will be used as “translators” between existing, specialized black box AI models and clinician-users, creating images to illustrate the what the original AI “perceives” about disease progression (which may incorporate irrelevant or incorrect information), allowing the clinician to diagnose and correct errors through a better understanding of their origins.
The intermediary VLLM will be used to create guidance on the internal operation of existing medical AI systems, suitable for both developers and clinicians to identify problems, debug, and improve performance. The approach will be evaluated using large-scale medical datasets and further tested via related, real-world tasks (such as breast cancer risk prediction).
Assistant Professor Kayhan Batmanghelich is the recent recipient of a collaborative $3.1M competitive renewal R01 grant from the NIH’s National Heart, Lung, and Blood Institute, for AI-driven research on Chronic Obstructive Pulmonary Disease (COPD). He received an inaugural Google Academic Research Award in Fall 2024, and is the founder of READE.ai, a start-up using real-time ML to evaluate complications during surgeries. He joined BU ECE in 2023.