Posted Tuesday 26 September 2017
A new artificial intelligence system that can create 3D selfies of people's faces from a single photograph has been developed by experts from Kingston University and the University of Nottingham.
Dr Vasileios Argyriou, from Kingston University's Faculty of Science, Engineering and Computing, was part of a team of researchers that trained a computer system to recognise features in a flat image and reconstruct them into an accurate three dimensional model.
The breakthrough was made possible through a form of machine learning called deep learning, which uses artificial neural networks to mimic the way the brain makes connections between pieces of information without being specifically programmed.
The team has now created an online app using the software, allowing anyone to upload a colour image of a face that is quickly processed and transformed into a 3D model – with more than 430,000 users having tried it out so far.
The study demonstrates the potential of deep learning to make real advances in the field of computer science, according to Dr Argyriou.
"What's really impressive about this technique is how it has made the process of creating a 3D facial model so much simpler," he said.
"Going from 2D to 3D has previously involved a number of stages that we've been able to bypass – we don't have to select specific features, or use morphable models.
"We give the system a certain amount of data and it then learns in a similar way to how we process seemingly unconnected information – allowing it to draw conclusions for what a flat image of a person's face would look like in three dimensions."
Dr Argyriou worked with a team from the University of Nottingham led by PhD student Aaron Jackson and Dr Yorgos Tzimiropoulos on the study – called Large Pose 3D Face Reconstruction from a Single Image via Direct Volumetric CNN Regression. The researchers will present their findings at the International Conference on Computer Vision in Venice next month.
The team used a Convolutional Neural Network (CNN) to train the computer system using a database of 80,000 2D images and corresponding 3D models. The information allows the CNN to reconstruct facial geometry from a single image – even making a good guess at non-visible parts of the face based on what it has learned.
The novelty of their work was the simplicity of the approach, according to Dr Yorgos Tzimiropoulos, from the University of Nottingham, which he said had allowed the team to bypass the complex pipelines typically used by other techniques.
This new deep learning approach to 3D modelling could be used in a variety of fields, from detecting faces in crowds to creating personalised avatars in future computer games, Dr Vasileios Argyriou added.
"You could use this technology for security purposes, where a still image of a face could be quickly scanned and a 3D image created that could help identify people in crowds from different angles," he explained. "It could also be used in animated films and for special effects in movies, as well as various medical applications.
"The system we've created is already very accurate, but we're now aiming to improve it further to be able to process the subtler details of the face – such as facial hair, glasses, moles and freckles."