art.faces - AI sculpts 3D faces from famous paintings
Exploration on face detection and 3D reconstruction using the "Large Pose 3D Face Reconstruction from a Single Image via Direct Volumetric CNN Regression" code by Aaron S. Jackson, Adrian Bulat, Vasileios Argyriou and Georgios Tzimiropoulos.
We selected 8 Famous Paintings and let the Convolutional Neural Network (CNN) perform a direct regression of a volumetric representation of the 3D facial geometry from a single 2D image.
Mona Lisa
Leonardo da Vinci
c. 1503–06, perhaps continuing until c. 1517
Oil on poplar panel
Subject: Lisa Gherardini
77 cm × 53 cm
Musée du Louvre, Paris
Girl with a Pearl Earring
Meisje met de parel
Johannes Vermeer
c. 1665
Oil on canvas
44.5 cm × 39 cm
Mauritshuis, The Hague, Netherlands
The Birth of Venus
Nascita di Venere
Sandro Botticelli,
The Birth of Venus (c. 1484-86).
Tempera on canvas.
172.5 cm × 278.9 cm.
Uffizi, Florence
Las Meninas
Diego Velázquez
1656
Oil on canvas
318 cm × 276 cm
Museo del Prado, Madrid
Self-Portrait
Vincent van Gogh
September 1889
Oil on canvas
65 × 54 cm
Musée d'Orsay, Paris.
This may have been Van Gogh's last self-portrait.
Laughing Cavalier
Frans Hals
1624
oil on canvas
83 cm × 67.3 cm
Wallace Collection, London
American Gothic
Grant Wood
1930
Oil on beaverboard
78 cm × 65.3 cm
Art Institute of Chicago
The Night Watch
Rembrandt van Rijn
1642
Oil on canvas
363 cm × 437 cm
Rijksmuseum, Amsterdam
The technology behind:
Code available on GitHub
Test your images with this demo,
This tech is fantastic and amazing, but same as with DeepFakes, it should be handled with care… Once we can generate accurate 3D representations of anyone with minimal input data, there’s the possibility to use that 3D output as digital identity theft.
As well questioned in this VentureBeat article :
So what happens when the technology further improves (which it will) and becomes accessible to marketers and brands (which it always does)? Imagine a casting call where a dozen actors are digitally and convincingly superimposed on a stand-in model prior to engaging the actors in real life.
We can imagine a scenario where videos are created using someone’s face (and body?) to make them do whatever they want, without consent.
On the other hand, AI is getting so smart, that in some cases, it doesn’t need a real input to go wild and imagine possible realities.
In this paper by NVIDIA, a generative adversarial network is able to picture imaginary celebrities.
Particularly in the 3D space, we see a huge potential for this kind of technology.
