A Visualization-Based Framework for Classifier Evaluation

Abstract

Classifier evaluation can be viewed as a problem of analyzing high- dimensional data. The performance measures currently used are but one class of projections—typically one-dimensional ones—that can be applied to these data. On the other hand, the projection approaches used in the analysis of high-dimensional data are typically intended for visualization and are, therefore, projected into a two-dimensional space. If applied to classifier evaluation, they would yield two advantages: 1) A quick and easy way for human-beings to assess classifier performance results; 2) the establishment of two rather than one relationships among classifiers: their ranking with respect to an ideal classifier and their comparison to one another.

In this talk, I will present our visualization-based framework for classifier evaluation and show how 1) this framework can be used to uncover the deep structure of real domains when used with artificial data; 2) it can be used to select the classifiers to include in an ensemble approach given a pool of classifiers.

The talk will also give a quick overview of the book on machine learning evaluation that I have recently co-written ("Evaluating Learning Algorithms", by Nathalie Japkowicz and Mohak Shah. Cambridge University Press, 2011)

Biography:

Nathalie Japkowicz is a Professor of Computer Science in the School of Information Technology and Engineering at the University of Ottawa, currently on sabbatical in the Department of Computer Science at Tufts University. She received her Ph.D. from Rutgers University, her M.Sc. from the University of Toronto and her B.Sc. from McGill University. Along with Machine Learning evaluation, her research interests include one-class learning, the class imbalance problem and learning in the presence of concept drifts.