ECE PhD Prospectus Defense: Shantanu Ghosh

ECE PhD Prospectus Defense: Shantanu Ghosh

Title: Interpretable Medical AI with Vision-Language Alignment

Presenter: Shantanu Ghosh

Advisor: Professor Kayhan Batmanghelich

Chair: Professor Brian Kulis

Committee: Professor Kayhan Batmanghelich, Professor Brian Kulis, Professor Wenchao Li, Professor Clare Poynton.

Google Scholar Profile: https://scholar.google.com/citations?user=U_s5k_oAAAAJ&hl=en

Abstract: Deep learning achieves state-of-the-art accuracy in medical imaging. Yet, black-box models suffer from systematic failures on critical patient subgroups. These models lack the transparency required for clinical diagnosis and error rectification. This thesis establishes a comprehensive framework to bridge the gap between predictive power and clinical trust.

Aim~1 explains black-box predictions using a predefined clinical concept bank. Specifically, it carves out a mixture of concept-based interpretable models from a pretrained black box, where each model explains a distinct subset of data using First-Order Logic (FOL) rules. These symbolic rules remain invariant across domains. Aim~1 exploits this symbolic invariance to transfer the interpretable models to an unseen domain with minimal training data, avoiding end-to-end black-box retraining. Next, Aim~2 eliminates the requirement for expensive, predefined concept banks with free-form radiology text. It introduces language-driven slice discovery that leverages vision--language alignment to identify subgroups where models fail and uses large language models to generate explicit, testable hypotheses about failure mechanisms; it further extends this framework with causal reasoning to validate and refine these hypotheses. Finally, Aim~3 proposes Mammo-FM, the first mammography-specific vision--language foundation model pre-trained on large-scale mammogram--report pairs. Leveraging the vision-language alignment of Mammo-FM, Aim~3 transforms the state-of-the-art risk black box models risk predictors \eg MIRAI, into interpretable risk predictors by grounding the risk scores in radiology report sentences. Overall, this thesis establishes interpretability as a practical mechanism for diagnosing systematic mistakes, improving trust, and advancing equitable medical AI.