Light-field camera plus VCSEL spots smile from frown

Share this on social media:

Face expression reading based on MLP classification from 3D depth maps and 2D images obtained by NIR-LFC. Credit: KAIST

Researchers from the Korea Advanced Institute of Science and Technology (KAIST) have optimised 3D light-field image reconstruction of faces using a VCSEL light source.

The team, led by Professors Ki-Hun Jeong and Doheon Lee from KAIST, trained a sub-miniature, near-infrared light-field camera with a vertical-cavity surface-emitting laser (VCSEL) to recognise face expressions using AI.

Unlike a conventional camera, a light-field camera contains a micro-lens array in front of the image sensor, which makes the camera small enough to fit into a smartphone, while allowing it to acquire the spatial and directional information of the light with a single shot.

The technique has received attention as it can reconstruct images in a variety of ways including multi-views, refocusing, and 3D image acquisition, giving rise to many potential applications.

However, the optical crosstalk between shadows caused by external light sources in the environment and the micro-lens has limited existing light-field cameras from being able to provide accurate image contrast and 3D reconstruction.

The researchers applied an NIR VCSEL to stabilise the accuracy of 3D image reconstruction, which previously depended on environmental light. When an external light source is shone on a face at 0-, 30-, and 60-degree angles, the light-field camera reduces 54 per cent of image reconstruction errors.

Additionally, by inserting a light-absorbing layer for visible and NIR wavelengths between the micro-lens array, the team could minimise optical crosstalk while increasing the image contrast by 2.1 times.

Through this technique, the team could overcome the limitations of existing light-field cameras and was able to develop an NIR-based light-field camera, optimised for reconstructing 3D images of face expressions.

The face expressions in the acquired 3D images were distinguished through machine learning with an average of 85 per cent accuracy.

Professor Ki-Hun Jeong said the team’s sub-miniature light-field camera could be applied in various fields including mobile healthcare, field diagnosis, social cognition, and human-machine interactions.

Dr Alireza Rastegarpanah working with a cobot on the task of unbolting a lithium-ion battery. Credit: Dr Alireza Rastegarpanah

25 April 2022

Dr Alireza Rastegarpanah working with a cobot on the task of unbolting a lithium-ion battery. Credit: Dr Alireza Rastegarpanah

25 April 2022

The system assesses different parts, like these two generators. Credit: Fraunhofer IPK/Larissa Klassen

01 April 2022

The lab-based prototype lidar system that the research team built captures megapixel-resolution depth maps using a commercial digital camera. (Image credit: Andrew Brodhead)

31 March 2022