Kingston University expert helps develop app that uses artificial intelligence to create 3D selfies from photographs

Posted Tuesday 26 September 2017

A new artificial intelligence system that can create 3D selfies of people's faces from a single photograph has been developed by experts from Kingston University and the University of Nottingham.

Dr Vasileios Argyriou, from Kingston University's Faculty of Science, Engineering and Computing, was part of a team of researchers that trained a computer system to recognise features in a flat image and reconstruct them into an accurate three dimensional model.

The breakthrough was made possible through a form of machine learning called deep learning, which uses artificial neural networks to mimic the way the brain makes connections between pieces of information without being specifically programmed.

The team has now created an online app using the software, allowing anyone to upload a colour image of a face that is quickly processed and transformed into a 3D model – with more than 430,000 users having tried it out so far.

The study demonstrates the potential of deep learning to make real advances in the field of computer science, according to Dr Argyriou.

"What's really impressive about this technique is how it has made the process of creating a 3D facial model so much simpler," he said.

"Going from 2D to 3D has previously involved a number of stages that we've been able to bypass – we don't have to select specific features, or use morphable models.

 "We give the system a certain amount of data and it then learns in a similar way to how we process seemingly unconnected information – allowing it to draw conclusions for what a flat image of a person's face would look like in three dimensions."

The artificial intelligence system has been trained to quickly create a 3D model of person's face from a single uploaded photograph (left).

Dr Argyriou worked with a team from the University of Nottingham led by PhD student Aaron Jackson and Dr Yorgos Tzimiropoulos on the study – called Large Pose 3D Face Reconstruction from a Single Image via Direct Volumetric CNN Regression. The researchers will present their findings at the International Conference on Computer Vision in Venice next month.

The team used a Convolutional Neural Network (CNN) to train the computer system using a database of 80,000 2D images and corresponding 3D models. The information allows the CNN to reconstruct facial geometry from a single image – even making a good guess at non-visible parts of the face based on what it has learned.

The novelty of their work was the simplicity of the approach, according to Dr Yorgos Tzimiropoulos, from the University of Nottingham, which he said had allowed the team to bypass the complex pipelines typically used by other techniques.

This new deep learning approach to 3D modelling could be used in a variety of fields, from detecting faces in crowds to creating personalised avatars in future computer games, Dr Vasileios Argyriou added.

"You could use this technology for security purposes, where a still image of a face could be quickly scanned and a 3D image created that could help identify people in crowds from different angles," he explained. "It could also be used in animated films and for special effects in movies, as well as various medical applications.

"The system we've created is already very accurate, but we're now aiming to improve it further to be able to process the subtler details of the face – such as facial hair, glasses, moles and freckles."

Latest news stories

Kingston University in top 15 in Europe for gender equality of staff and students in new Times Higher Education teaching excellence ranking

Kingston School of Art illustration animation students draw high praise from judges to win Royal Television Society student awards

Kingston University students step into the spotlight as International Youth Arts Festival celebrates its 10th anniversary

Kingston University appoints Professor Jill Schofield as Pro Vice-Chancellor and Dean of its new Faculty of Business and Social Sciences

Contact us

General enquiries:

Journalists only: