August 2020: We are compiling summaries of state-of-the-art research in ethics at the frontier of technology, following the theme of our 2019 Susilo Symposium. Today, we review insights on understanding and utilizing medical Artificial Intelligence by Romain Cadario (Erasmus University), Chiara Longoni & Carey K. Morewedge (Boston University).<\/p>\n
Medical Artificial intelligence (AI) such as algorithmic-based healthcare services are rapidly growing with various applications ranging from detection of skin cancer<\/a>, emergency department triage<\/a>, to diagnoses of COVID-19 from chest X-rays<\/a>. Medical artificial intelligence is cost-effective, scalable, and often outperforms human providers<\/a>. Unfortunately, patients exhibit resistance to medical AI<\/a>.<\/p>\n In this research, Cadario, Longoni and Morewedge show that the perception that medical AI is a \u201cblack box\u201d creates a barrier to its utilization. People erroneously believe they understand medical decisions made by human providers, and that only decisions made by algorithmic providers are a blackbox. In fact, the authors show that both are a blackbox, and this illusion of explanatory depth<\/a> impairs the utilization of algorithmic relative to human providers. Fortunately, the authors show that interventions can increase subjective understanding of algorithmic decision processes, which increases willingness to utilize algorithmic healthcare providers at no expense to the utilization of human providers.<\/p>\n In one experiment, participants were assigned to either a human provider or an algorithmic provider condition in the context of skin cancer detection. Participants read that \u201cA dermatologist [algorithm] will examine the scans of your skin to identify cancerous skin lesions,\u201d and then completed a measure of subjective understanding (\u201cTo what extent do you understand how a dermatologist [algorithm] examines the scans of your skin to identify cancerous skin lesions?\u201d). To test the degree to which this rating was illusory, participants then generated an open-ended explanation of the provider\u2019s decision process. After completing the explanation, participants provided a second rating of their subjective understanding of a dermatologist\/algorithmic dermatologist\u2019s decision process. Comparing the results from pre to post-explanation, the authors found a significant decrease in subjective understanding for human providers, but not for algorithmic providers. In other words, people were more aware of their limited understanding of medical decisions made by algorithmic than human providers.<\/p>\n Next, Cadario and colleagues tested an intervention to increase subjective understanding and utilization of algorithmic healthcare providers. Participants were assigned to one of four experimental conditions (control & human provider, control & algorithmic provider, intervention & human provider, intervention & algorithmic provider). Using the same context of skin cancer detection, participants in the intervention condition first rated their subjective understanding, and then read supplementary information that described how doctors [algorithms] diagnose skin cancer based on photographs of moles. The information was presented in a single diagram using the ABCD (Asymmetry, Border, Color, Diameter) framework. Compared to a control condition (without supplementary information), the intervention lead to an increase in subjective understanding of algorithmic providers, which in turn increased utilization intentions of algorithmic providers.<\/p>\n