Sticker camera addresses IoT imaging

Share this on social media:

Tags: 

Greg Blackman reports on CSEM's Witness IOT camera, an ultra-low power imager that can be deployed as a sticker. Dr Andrea Dunbar presented the technology at the Image Sensors Europe conference in London in March

As demonstrations of Internet-of-Things devices go, Dr Andrea Dunbar’s unobtrusive camera stuck to the plinth from which she gave her presentation at Image Sensors Europe in London on 13 March added a nice finishing touch to her description of technology designed to be positioned anywhere and everywhere.

Dunbar, who is working for Swiss research organisation CSEM, pointed out the tiny camera at the end of her talk: ‘I put one here,’ she said, ‘I don’t know whether you noticed it. So you’re currently being videoed.’

CSEM has developed the Witness IOT camera for security and other Internet-of-Things (IoT) applications. Its unique selling point is it can be deployed as a sticker, and shows how an imaging device can be optimised to keep power consumption and cost low enough to act as an autonomous, smart, and useful data logger.

Witness combines an ultra-low power imager, photovoltaic power supply, and wireless reporting. The electronics are thin (200µm) and flexible; it’s built using low-cost components, although the imager was developed specifically for the project, and the camera operates based on motion detection, only recording when activity is happening in the scene.

Security tasks, applications in healthcare – to trigger alerts if elderly people fall, for instance – and people counting in buildings, are all potential uses of this technology.

Dunbar explained the requirements of the Witness project: ‘We need a low-power imager that will work in general lighting conditions, which are quite extreme, so it needs to have a high dynamic range. It also needs embedded intelligence, and to do some wireless reporting. From the data, you will get a feedback loop into actuation or reporting. All of this needs to be under a certain power [consumption].’

The camera, called Ergo, is QVGA resolution (320 x 320 pixels); it has 120dB dynamic range, 107° field of view using a special lens, and detects motion on sub-sampled images.

‘We first tried to buy an imager,’ recalled Dunbar, ‘because we believe in buying low-cost hardware, but we couldn’t find an imager with sufficient dynamic range coupled with resolution and power budget, so we decided to make our own called Ergo.’

The 10-bit Ergo imager has logarithmic encoding; it has an on-chip frame memory and time-to-voltage integration, and consumes700µW of power at 10fps. The imager is able to read out regions of interest and uses a supply voltage of 1.8V.

Ergo acquires an image and then performs background subtraction to determine whether there’s movement in the scene. If there’s no motion, the imager returns to sleep mode to save power until it is time to acquire the next frame. If there is motion only the change in the scene is compressed and stored in the onboard flash memory. It runs at 1fps. ‘Only a change [in the scene] requires a response, and this is absolutely key to the low-power part [of the device],’ Dunbar said.

‘This is all achieved using rather poorly optimised COTS components at the moment,’ she added. ‘We believe we can still drastically reduce the processing consumption of the hardware.’

The team at CSEM also wants to implement machine learning on the device to optimise processing. ‘We need embedded intelligence on our [IoT] devices, because we’re not going to be able to send videos or photos with the power budget we have,’ Dunbar explained. A 20 megapixel image has 1 million bytes of information, which is a huge amount to transmit. The idea is that a neural network would reduce the image data to just a few bytes containing only the information needed to make a decision.

‘Intelligence at the edge … minimises the quantity of data stored and exchanged – we get more accurate information,’ Dunbar said. ‘It also makes it more secure and more private – we’re not transmitting images of the scene. Embedded intelligence is essential to have an autonomous IoT design.’

However, a key issue is that the software has to be able to perform image capture, analysis, and compression with very scarce resources.

Dunbar summed up: ‘The main achievement [of Witness] was to balance the whole system; we had to take into account that we had no motion, normal motion, or intensive motion, which would require different quantities of energy compared to what is gathered by the photovoltaic cell.’

The solar cell is not a standard photovoltaic device, Dunbar noted. CSEM has knowhow in photovoltaics and the module used in Witness is optimised for low-light conditions. However, the cost is still comparable to what’s available on the market, she said. To make an IoT device truly autonomous using solar power means having to live with a power budget of between 10µW and 100mW, she added.

CSEM’s roadmap for Witness is to increase the resolution to VGA. ‘We’ve done some trials where we’re still going to be in the 1mW range for 10fps,’ Dunbar said. ‘This is an optimised system, and IoT systems are going to remain optimised if we want them to work. So we will be optimising either for people counting or detection, and we will be using machine learning hardware to extract information from the scenes.’

Related article

Infrared, China, and 3D stacking: CMOS trends - Greg Blackman reports from the Image Sensors Europe conference in London, which took place from 13 to 14 March

Other tags: 
Company: 

Related analysis & opinion

22 February 2019

Ron Low, Framos head of sales Americas and APAC, reports from Framos Tech Days at Photonics West in San Francisco where Sony Japan representatives presented image sensor roadmap updates

20 February 2019

Jeff Bier, founder of the Embedded Vision Alliance, discusses the four key trends driving the proliferation of visual perception in machines

19 February 2019

Greg Blackman reports on CEA Leti's new image sensor, shown at Photonics West, which contains onboard processing and is able to image at 5,500 frames per second

15 November 2018

Greg Blackman reports on the buzz surrounding embedded vision at the Vision Stuttgart trade fair, which took place from 6 to 8 November

26 October 2018

Arndt Bake, chief marketing officer of Basler, looks back at the history of machine vision to see what lessons can be applied to the emerging embedded vision market

Related features and analysis & opinion

29 March 2019

Greg Blackman reports from Embedded World, in Nuremberg, where he finds rapid progress in technology for imaging at the edge

22 February 2019

Ron Low, Framos head of sales Americas and APAC, reports from Framos Tech Days at Photonics West in San Francisco where Sony Japan representatives presented image sensor roadmap updates

20 February 2019

Jeff Bier, founder of the Embedded Vision Alliance, discusses the four key trends driving the proliferation of visual perception in machines

19 February 2019

Greg Blackman reports on CEA Leti's new image sensor, shown at Photonics West, which contains onboard processing and is able to image at 5,500 frames per second

15 November 2018

Greg Blackman reports on the buzz surrounding embedded vision at the Vision Stuttgart trade fair, which took place from 6 to 8 November

31 October 2018

As the worldwide machine vision market continues to expand – with new trends emerging and new elements coming into play that could impact existing business models – companies are searching for those ever-important opportunities to stimulate growth.

One such trend is almost certainly embedded vision, although the technology behind it is not new, as Mark Williamson, managing director at Stemmer Imaging, noted: ‘Embedded vision is a big topic. However, it has been here a long time, because every smart camera that you buy is an embedded vision system.’

31 October 2018

Qualcomm Technologies' Snapdragon board is designed for mobile devices, but can be used to create other embedded vision systems. Credit: Qualcomm Technologies

Embedded computing promises to lower the cost of building vision solutions, making imaging ubiquitous across many areas of society. Whether this turns out to be the case or not remains to be seen, but in the industrial sector the G3 vision group, led by the European Machine Vision Association (EMVA), is preparing for an influx of embedded vision products with a new standard.

26 October 2018

Arndt Bake, chief marketing officer of Basler, looks back at the history of machine vision to see what lessons can be applied to the emerging embedded vision market