Effective AI Decision Support: Interpretability and Interaction

Finale Doshi-Velez
Professor
Harvard University
Interpretability has been proposed as a way to improve the human+AI performance in the context of AI decision support. If a human can understand the model, or at least the part of the model responsible for the decision, the thinking goes, then they can make better choices about how to use the model effectively. This thinking relies on two critical assumptions: First, that we can present the model or the most relevant parts of the model. Second, that people will use that information effectively to make better choices. In this talk, I’ll discuss how yes, there *is* a path towards models that help people make better decisions — but only if we think carefully about those two assumptions.
When the decision involves multiple steps, like an RL policy, how can we create something that’s compact enough for a human to inspect and understand? What are the most important elements to expose? When there is a lot of information involved in creating a strategy, how can we make sure that the human knows what they need to know in order to make good decisions? I’ll share how we are approaching interpretability and interaction to address questions like these in two settings: assisting doctors in the ICU and assisting humanitarian crisis negotiators. I’ll use these cases to highlight open questions in how to make machines that effectively augment rather than replace people.
Finale Doshi-Velez is a Herchel Smith Professor in Computer Science at the Harvard Paulson School of Engineering and Applied Sciences. She completed her MSc from the University of Cambridge as a Marshall Scholar, her PhD from MIT, and her postdoc at Harvard Medical School. Her interests lie at the intersection of reinforcement learning, interpretability, and probabilistic methods to improve human+AI discovery and decision-making.