Artificial intelligence enables image fusion technology

Share this on social media:

Artificial intelligence is being used in a new multi-modal image fusion technology that automatically combines the best parts of images from different sources. The technology, jointly developed by NEC Corporation and the Tokyo Institute of Technology, can combine visible images taken by standard cameras with non-visible images taken by specialised devices, such as thermal or terahertz cameras.

Specialised non-visible imaging devices, such as infrared and x-ray cameras, can now be used in an extended range of environments, including night time monitoring under severe weather conditions. However, these cameras tend to provide images with poor resolution and quality in comparison to images taken of visible subjects. To overcome this, non-visible images are often compared or combined with clearer visible images of the same subject; however, in the past this has lead to abnormalities or hazards within the images being overlooked.

'This technology eliminates the need for such manual work, using artificial intelligence to effectively and automatically combine images taken by different cameras,’ said Professor Masatoshi Okutomi of the Tokyo Institute of Technology. ‘This also increases visibility by actively utilising the strong points of each visible image and non-visible image, even when the images are difficult to visualise.’

The artificial intelligence carries out detailed examinations of each image and assess the degree of visibility of each part. The best areas of each image are then automatically extracted and combined in a multi-modal image, taking into account environmental characteristics such as brightness, light direction and obstacles.

‘As this technology enables instant visual clarification, even under harsh conditions, it allows users to make well informed evaluations,’ commented Akio Yamada, general manager of Data Science Research Laboratories at NEC. ‘For example, it can be applied to monitoring systems to assist with night time observations, or for infrastructure inspection devices to improve the detection of interior and exterior abnormalities, such as cracking.’ 

The artificial intelligence also analyses and detects abnormalities and hazards in non-visible camera images while simultaneously avoiding image breakdowns, such as clipped highlights and crushed blacks. Image analysis is a common use of artificial intelligence and deep learning. In April, system-on-chip manufacturer Socionext and Japanese AI software company Soinn presented results of a trial using deep learning algorithms to assist technicians and detect human errors in medical image handling.

NEC and Tokyo Tech presented the technology on 7 June at the 23rd Symposium on Sensing via Image Information in Yokohama City, Japan.

Related news

Credit: Nataly Reinch _ Shutterstock.com

27 May 2021

The vision devices use Sony's IMX500 sensor, which is able to run AI algorithms on the chip to provide real-time information about free parking spaces and other transport data

Credit: CEA; metamorworks-shutterstock - 2021/01

25 May 2021

The face recognition imager consumes 10,000 times less energy than a typical camera and processor. CEA-Leti is working with STMicroelectronics on the imager

20 April 2021

The Kria K26 SOM is built on top of the Zynq UltraScale+ MPSoC architecture. It has 4GB of DDR4 memory and 245 IOs for connecting sensors

14 April 2021

The platform integrates the Nvidia Jetson TX2NX module with an edge-to-cloud software stack including AWS Panorama Device SDK

Flir Firefly DL. Credit: Flir

26 March 2021

Customers can create deep learning models using Neurala’s Brain Builder software and then upload these to a Flir Firefly DL camera

02 March 2021

Teledyne e2v and Yumain aim to make AI technology more accessible by addressing challenges associated with AI