Researchers succeed in teaching deep neural networks to synthesise photographs

Share this on social media:

A research team at the Radboud University in the Netherlands are using deep neural networks to convert hand-drawn face sketches into photorealistic images. The success and accuracy of this process could lead it to have applications in forensics, converting sketches of potential suspects into realistic photographs.

By using a set of 200,000 facial images, the researchers produced a set of training data for the 11-layer network by converting the images into grayscale sketches and line drawings through standard image processing algorithms. The network was then set to the task of reversing the process to turn a generated sketch back into the photograph of a face. After successfully working with the sample data, the researchers used the now-trained network to perform the same process with a new set of images. The results were promising. ‘Synthesised face images matched the ground truth photographs closely, and persons in the images were easily recognisable in most cases,’ say the team in the research paper. They found that the best results were produced when the network was working with a line sketch (produced by an algorithm) as the starting image. ‘We found that the line model performed impressively in terms of matching the hair and skin colour of the individuals even when the line sketches did not contain any colour information’ the team commented. 

The next task faced by the trained network was synthesising photorealistic images from a hand drawn sketches, rather than those generated by an algorithm. The technique proved once again a success. ‘We found that the line sketch inversion model can solve this inductive transfer task almost as good as it can solve the task that it was trained on. Once again, the model synthesised photorealistic face images’. With the results, it was easy for the team to recognise the advantages of this model over those that came previously. ‘Hair texture and style, which posed a problem in most previous studies, was very well handled by our CSI model.’ 

The Radboud University research team believe the process has uses in both forensics and fine arts. They see the network being used to compare sketches of potential suspects in crime investigations to a database of photographs in order to identify them, and using it to turn historic portraits of artists into photographs, producing what would be the first photographic records from that time. 

Further Information:

Original story 

Recent News

29 July 2020

The Perseverance rover contains 19 cameras, including seven scientific instruments. It will analyse the climate and geology of Mars, looking for signs of past life, as well as monitoring the Martian atmosphere

02 July 2020

Norwegian seafood firm, Lerøy, has installed hyperspectral cameras on processing lines to sort fish. The system is able to measure the amount of blood in white fish, which gives a grade of quality

09 June 2020

Hyperspectral imaging is being used in a research programme at hospitals in Maryland and New York to investigate the prognostic value of skin findings associated with Covid-19 infection

27 May 2020

The composite picture of The Night Watch, made of 528 exposures stitched together digitally, makes it possible to zoom in on individual brushstrokes and even particles of pigment in the painting