November 18, 2016, Pradeep Ravikumar, Carnegie Mellon University

Friday, November 18, 2016, 2pm-3pm
8 St. Mary’s Street, PHO 404/428
Refreshments at 1:45pm

Pradeep Ravikumar
Carnegie Mellon University

Classification with General Performance Metrics

In the classical problem of classification, the goal is to learn from data a classifier that predicts a categorical response given input covariates. Fundamental questions here are: what is the classifier that is optimal with respect to some classification performance metric, and what are computationally tractable algorithms to estimate this optimal classifier? Most of the work that answer these questions have focused on the zero-one classification performance metric, which assigns a cost of zero if the categorical response is predicted correctly, and one if not.

But in many modern settings, performance metrics of interest require more complex tradeoffs between true and false positives and negatives. We first focus on the simple case of binary classification, where the response is binary, and answer the fundamental questions above for a fairly large family of performance metrics we call fractional linear metrics, that include not just the zero-one loss, but many other well known binary classification metrics used in practice such as classification accuracy, AM measure, F-measure and the Jaccard similarity coefficient as special cases. We then extend our results to more complex settings involving multivariate losses that entail the prediction of multiple categorical responses simultaneously. For these varied classification performance metrics, we not only characterize the set of optimal classifiers, but also provide practical algorithms that estimate these optimal classifiers with strong theoretical guarantees.

Joint work with Nagarajan Natarajan, Sanmi Koyejo, Inderjit Dhillon.

Pradeep Ravikumar is an Associate Professor in the Machine Learning Department, School of Computer Science at Carnegie Mellon University. His thesis has received honorable mentions in the ACM SIGKDD Dissertation award and the CMU School of Computer Science Distinguished Dissertation award. He is a Sloan Fellow, a Siebel Scholar, a recipient of the NSF CAREER Award, and was Program Chair for the International Conference on Artificial Intelligence and Statistics (AISTATS) in 2013.

Faculty Host: Prakash Ishwar
Student Host:  Tingting Xu