Exploration on face detection and 3D reconstruction using the "Large Pose 3D Face Reconstruction from a Single Image via Direct Volumetric CNN Regression" code by Aaron S. Jackson, Adrian Bulat, Vasileios Argyriou and Georgios Tzimiropoulos.

link: https://www.cunicode.com/works/artfaces Using the demo from the paper "Large Pose 3D Face Reconstruction from a Single Image via Direct Volumetric CNN Regression", we let the AI sculpt 3D faces from famous artworks.

We selected 8 Famous Paintings and let the Convolutional Neural Network (CNN) perform a direct regression of a volumetric representation of the 3D facial geometry from a single 2D image.

01 Mona_Lisa,_by_Leonardo_da_Vinci,_from_C2RMF_retouched.jpg — Mona Lisa

Leonardo da Vinci
c. 1503–06, perhaps continuing until c. 1517
Oil on poplar panel
Subject: Lisa Gherardini
77 cm × 53 cm
Musée du Louvre, Paris

03 Meisje_met_de_parel.jpg — Girl with a Pearl Earring

Meisje met de parel
Johannes Vermeer
c. 1665
Oil on canvas
44.5 cm × 39 cm
Mauritshuis, The Hague, Netherlands

05 Sandro_Botticelli_-_La_nascita_di_Venere_-_Google_Art_Project_-_edited.jpg — The Birth of Venus

Nascita di Venere
Sandro Botticelli,
The Birth of Venus (c. 1484-86).
Tempera on canvas.
172.5 cm × 278.9 cm.
Uffizi, Florence

Las Meninas

Diego Velázquez
1656
Oil on canvas
318 cm × 276 cm
Museo del Prado, Madrid

02 Vincent_van_Gogh_-_Self-Portrait_-_Google_Art_Project.jpg — Self-Portrait

Vincent van Gogh
September 1889
Oil on canvas
65 × 54 cm
Musée d'Orsay, Paris.
This may have been Van Gogh's last self-portrait.

04 Cavalier_soldier_Hals-1624x.jpg — Laughing Cavalier

Frans Hals
1624
oil on canvas
83 cm × 67.3 cm
Wallace Collection, London

06 Grant_Wood_-_American_Gothic_-_Google_Art_Project.jpg — American Gothic

Grant Wood
1930
Oil on beaverboard
78 cm × 65.3 cm
Art Institute of Chicago

08 The_Nightwatch_by_Rembrandt.jpg — The Night Watch

Rembrandt van Rijn
1642
Oil on canvas
363 cm × 437 cm
Rijksmuseum, Amsterdam

The technology behind:

Code available on GitHub
Test your images with this demo,

This tech is fantastic and amazing, but same as with DeepFakes, it should be handled with care… Once we can generate accurate 3D representations of anyone with minimal input data, there’s the possibility to use that 3D output as digital identity theft.

As well questioned in this VentureBeat article :

So what happens when the technology further improves (which it will) and becomes accessible to marketers and brands (which it always does)? Imagine a casting call where a dozen actors are digitally and convincingly superimposed on a stand-in model prior to engaging the actors in real life.

We can imagine a scenario where videos are created using someone’s face (and body?) to make them do whatever they want, without consent.

On the other hand, AI is getting so smart, that in some cases, it doesn’t need a real input to go wild and imagine possible realities.

In this paper by NVIDIA, a generative adversarial network is able to picture imaginary celebrities.

Particularly in the 3D space, we see a huge potential for this kind of technology.