PhD Seminar Series: WinoReferral: A Benchmark to Evaluate LLM Responses to Prompts Implying Mental Health Disorders

The use of LLMs has contributed to serious mental health conditions among a subset of users, most alarmingly cases of delusional thinking and suicide. My talk will begin by surveying leading theories from psychology and human computer interaction that explain how LLM use interacts with user mental health. I will then present my research on evaluating LLM responses to prompts that imply a user has depression and discuss our current results, which indicate high variance in AI companies’ safety policies for referring users to mental health professionals. I will conclude by proposing policy changes that could limit the harms of LLMs on user mental health.