Wednesday, October 4, 2017

Reconstructing a Face from a Single Image

A free online tool lets you create a 3D reconstruction of a face from a single image.

Van Dyck's Portrait of Cornelis Van Der Geest in 3D
You can input a single photo or a painting. After it processes and outputs, you can drag the 3D model around with your mouse and see it in a variety of angles.

3D Alfred E. Neuman, thanks MAD Magazine
It's fun to try it out on a familiar face that's usually seen only from one angle, like Mad Magazine's Alfred E. Neuman.

The tool was created by computer vision scientists at the University of Nottingham using machine-learning software called a Convolutional Neural Network (CNN). 
"Our CNN works with just a single 2D facial image, does not require accurate alignment nor establishes dense correspondence between images, works for arbitrary facial poses and expressions, and can be used to reconstruct the whole 3D facial geometry (including the non-visible parts of the face) bypassing the construction (during training) and fitting (during testing) of a 3D Morphable Model."
(Free interactive tool) Face Reconstruction from a Single Image
(Scientific paper) Large Pose 3D Face Reconstruction from a Single Image via Direct Volumetric CNN Regression

Thanks, Geoff Charlewood


Mark Szymanski said...

I did this with Ingres' Louis-Francois Bertin Portrait. It was awesome to see how well the image translated into different perspectives - head facing right or looking down or from the below viewpoint. Each viewpoint gives a very different idea of the man. It's again instructive to see how an artist chooses a particular pose to help capture the essence of his subject. I know Ingres had really struggled to come up with this pose to capture the force of nature L. Bertin appears in the painting.

Leonid Ilyukhin said...

I also tried Bertin. And also Durer's self-portrait en face. The side views of the heads were quite similar - the same side-view template. Seems like either there is not enough information about the depth in the pictures or the program can't (yet) process such information as effectively as a human mind.