Appearance Decomposition for Image-based Reconstruction

Abstract

Image-based reconstruction systems are designed to accurately recover the three-dimensional shape of a scene from its two-dimensional images. Applications include visual inspection, reverse engineering, digital object archival, enrollment for 3D face/object recognition systems, and virtual/augmented reality. The reconstruction problem is ill-posed because images do not generally provide direct access to 3D shape. Instead, shape information is coupled with additional factors: illumination, pose and surface reflectance.

Most existing reconstruction methods obtain 3D shape by making assumptions about reflectance. For example, it is often assumed that surfaces are perfectly matte or diffuse (i.e., Lambertian.) The applicability of these methods is limited because the accuracy of the recovered shape can be compromised when the underlying assumptions are violated.

In this talk I present two techniques for recovering 3D shape without making restrictive assumptions about surface reflectance. Both methods are based on the notion of appearance decomposition: by uncoupling some of the factors that determine an image (shape, reflectance, illumination and pose), we obtain more direct access to shape information, greatly simplifying the reconstruction problem. I demonstrate how these methods can be applied to a much broader class of surfaces, and discuss their applications.

For more information, see http://www.eecs.harvard.edu/~zickler/research.html