A Robust-EquitabLe Dependence Measure for Feature Selection (Aidong Ding - Northeastern University)

Dependence measure plays an important role in filter-based feature selection. To correctly identify important features with complex relationship in large data sets, we like the measure to be equitable (Reshef et al. Science, 2011): treating all types of functional relationships, linear and nonlinear, equally. We provides a theoretical treatment of equitability, including the self-equitability definition (Kinney and Atwal, PNAS 2014) and a new robust-equitablity definition. The robust copula dependence (RCD) measure based on $L_1$-distance of copula density is shown to be equitable under all equitability definitions. We also provide theoretical justification that RCD can be fundamentally easier to estimate than mutual information (MI), the recommended self-equitable measure in Kinney and Atwal. Numerical examples, on synthetic data sets and real data sets illustrate the effect of equitability in feature ranking and selection. Particularly, selection based on RCD can be more robust to varying sample size than selection through MI and other measures.