Special Joint Math/CS Colloquium: A Multilinear (Tensor) Framework for Computer Vision and Graphics

Abstract

Principal components analysis (PCA) is one of the most valuable results from applied linear algebra. It is used ubiquitously in all forms of data analysis - in data mining, biometrics, psychometrics, chemometrics, bioinformatics, computer vision, computer graphics, etc.- because it is a simple, non-parametric method for extracting relevant information through the dimensionality reduction of high-dimensional datasets in order to reveal hidden underlying variables. PCA is a linear method, however, and as such it has severe limitations when applied to real world data. We are addressing this shortcoming via multilinear algebra, the algebra of higher order tensors.

In the context of computer vision and graphics, we deal with natural images which are the consequence of multiple factors related to scene structure, illumination, and imaging. Multilinear algebra, offers a potent mathematical framework for explicitly dealing with multifactor image datasets. I will present two multilinear models that learn (nonlinear) manifold representations of image ensembles in which the multiple constituent factors (or modes) are disentangled and analyzed explicitly. Our nonlinear models are computed via two tensor decomposition, known as the N-mode SVD, which is an extension to tensors of the conventional matrix singular value decomposition (SVD) and through a generalization of conventional (linear) independent components analysis (ICA) called Multilinear Independent Components Analysis (MICA).

I will demonstrate the potency of our novel statistical learning approach in the context of facial image biometrics, where the relevant factors include different facial geometries, expressions, lighting conditions, and viewpoints. When applied to the difficult problem of automated face recognition, our multilinear representation, called TensorFaces (N-mode SVD) and Independent TensorFaces (MICA), yields significantly improved recognition rates relative to the standard PCA and ICA approaches. Recognition is achieved with our Multilinear Projection Operator.

In graphics, our image-based rendering technique, called TensorTextures, is a multilinear generative model that, from a sparse set of example images of a surface, learns the interaction between viewpoint, illumination and geometry, which determines surface appearance, including complex details such as self-occlusion and self shadowing. Our tensor algebraic framework is also applicable to human motion data, in order to extract "human motion signatures" that are useful in graphical animation synthesis and motion recognition.

Bio:M. Alex O. Vasilescu (www.media.mit.edu/~maov) was educated at MIT and the University of Toronto. Currently, she is a research scientist at MIT, Media Lab. She has done research at the MIT Artificial Intelligence Lab, IBM, Intel, Compaq, and Schlumberger corporations. She has published papers in computer vision and computer graphics, particularly in the areas of face recognition, human motion analysis/synthesis, image-based rendering, and physics-based modeling (deformable models). She has given several invited talks about her work and has three patents pending. Her face recognition research, known as TensorFaces, has been funded by the TSWG, the Department of Defense's Combating Terrorism Support Program. She has been named by MIT's Technology Review Magazine to their 2003 TR100 List of Top Young Innovators.