3D Shape Inference from Images using Deep Learning

The talk will cover two approaches to obtaining 3D shape from images. First, we introduce a deep Convolutional Neural Network (ConvNet) architecture that can generate depth maps given a single or multiple images of an object. The ConvNet is trained using a prediction loss on both the depth map and the silhouette. Using a set of sculptures as our 3D objects, we show that the ConvNet is able to generalize to new objects, unseen during training, and that its performance improves given more input views of the object. This is joint work with Olivia Wiles. Second, we use ConvNets to infer 3D shape attributes, such as planarity, symmetry and occupied space, from a single image. For this we have assembled an annotated dataset of 150K images of over 2000 different sculptures. We show that 3D attributes can be learnt from these images and generalize to images of other (non-sculpture) object classes. This is joint work with Abhinav Gupta and David Fouhey.