Reusing code in embedded vision applications

Share this on social media:

Tags: 

Migrating software code from a PC to an embedded system to run vision applications presents various challenges. Following a presentation he gave at Embedded World in Nuremberg, Germany last week, Frank Karstens, field application engineer at Basler, gives his advice on the task

A growing number of companies active in the machine vision field are beginning to recognise the benefits of an embedded approach. Most traditional machine vision systems are based on a classic PC setup: camera, cable and, in most cases, a Windows PC. Embedded vision systems offer the benefit of lower power consumption; they need less space and ultimately help the user to significantly reduce the cost of the system.

Nevertheless, there are a few challenges that need to be overcome to transfer existing software code from a classic PC setup to an embedded target, such as different operating systems, Windows versus hardware-specific Linux; different processor architectures, such as x86 versus ARM; or different camera interfaces, for example GigE versus MIPI CSI-2.

Well-defined standards with well-defined interfaces help to create the framework conditions for these differences. For a classic PC-based setup the industry has already found an answer through the GenICam standard, which was established in 2003.

GenICam – Generic Interface for Cameras – standardises camera configuration and image data transfer, and provides software developers with standard APIs. There are reference implementations with GenICam for various operating systems and processor architectures. On top of that, there are also camera manufacturer-specific, GenICam-based SDKs available, which make the camera APIs easier to use. The broader the choice of the SDKs’ supported operating systems, processor architectures and camera interface technologies, the greater the flexibility the user has when moving from one technology to another, and the easier it is to port existing code to a new target.

In typical, PC-based machine vision applications, GenICam helps to provide a stable interface that defines the specifics for camera or interface and allows plug-and-play functionality. Code written for a GigE camera by manufacturer A can easily be reused for a USB 3.0 camera by manufacturer B – the user only needs to make minor modifications.

Basler's Dart board camera with MIPI development kit

For embedded vision, however, things are different. There are a number of variables that need to be considered. The vision sensor is not necessarily a camera; it could be a camera module or even a bare CMOS sensor. In addition, there is an even greater choice of processing platforms than in the machine vision world. The different classic CPU architectures – such as x86, ARM, MIPS or PowerPC – have to compete with FPGA, GPU, DSP-based approaches and so on. Moving from one sensor or camera to another, or changing the processing system architecture, quite likely requires the software developer to rewrite significant parts of the vision software.

The possible challenges that may come up when migrating code from a non-embedded setup to an embedded one depend on the selected camera interface. The code for an embedded system with GigE or USB 3.0 is actually not that different from the code of a non-embedded approach. If the camera interfacing code for the non-embedded system was written for a GenICam-compliant API, and as long as the targeted processing platform provides these interfaces, the existing code can be reused without any modification when ported to an embedded target.

If the engineer wants to use the MIPI CSI-2 interface, this is not the case. In 2003, vendors of mobile devices or components formed the Mobile Industry Processor Interface (MIPI) Alliance, as it became more obvious that standards for connecting peripherals, including all kinds of sensors or displays, are required to speed up development and product release cycles. The CSI-2 specification – Camera Serial Interface, second generation – is today’s number one standard for connecting vision sensors or camera modules to mobile processors or SoCs.

MIPI CSI-2 will become the most important camera interface for embedded machine vision applications. The lack of standardised APIs like GenICam makes it difficult to reuse code that was written for other GenICam-compliant camera hardware.

Some camera vendors are now starting to put a MIPI CSI-2 driver and software stack under the hood of their existing vendor-specific, GenICam-based, camera SDK, which abstracts CSI-2 specifics. From the point of view of the user, such an API would look exactly like any other camera API from the same vendor and thus makes migration from another camera interface technology very easy. However, until the MIPI Alliance integrates GenICam into the CSI specification, this vendor-specific approach will remain a proprietary solution for specific camera/SoC combinations.

For the software developer this means finding a camera SDK that offer the broadest support for both non-embedded and embedded processing platforms, operating systems and interface technologies, including MIPI CSI-2. Having one unified camera API means it is possible to reuse significant amounts of existing code, and offers more flexibility to the user to move from one technology to another and to port existing code to the new target.

Related article:

Building embedded IR imaging devices: what to consider

Other tags: 

Related analysis & opinion

15 April 2019

Greg Blackman reports on CSEM's Witness IOT camera, an ultra-low power imager that can be deployed as a sticker. Dr Andrea Dunbar presented the technology at Image Sensors Europe in London in March

22 February 2019

Ron Low, Framos head of sales Americas and APAC, reports from Framos Tech Days at Photonics West in San Francisco where Sony Japan representatives presented image sensor roadmap updates

20 February 2019

Jeff Bier, founder of the Embedded Vision Alliance, discusses the four key trends driving the proliferation of visual perception in machines

19 February 2019

Greg Blackman reports on CEA Leti's new image sensor, shown at Photonics West, which contains onboard processing and is able to image at 5,500 frames per second

15 November 2018

Greg Blackman reports on the buzz surrounding embedded vision at the Vision Stuttgart trade fair, which took place from 6 to 8 November

Related features and analysis & opinion

15 April 2019

Greg Blackman reports on CSEM's Witness IOT camera, an ultra-low power imager that can be deployed as a sticker. Dr Andrea Dunbar presented the technology at Image Sensors Europe in London in March

29 March 2019

Greg Blackman reports from Embedded World, in Nuremberg, where he finds rapid progress in technology for imaging at the edge

22 February 2019

Ron Low, Framos head of sales Americas and APAC, reports from Framos Tech Days at Photonics West in San Francisco where Sony Japan representatives presented image sensor roadmap updates

20 February 2019

Jeff Bier, founder of the Embedded Vision Alliance, discusses the four key trends driving the proliferation of visual perception in machines

19 February 2019

Greg Blackman reports on CEA Leti's new image sensor, shown at Photonics West, which contains onboard processing and is able to image at 5,500 frames per second

15 November 2018

Greg Blackman reports on the buzz surrounding embedded vision at the Vision Stuttgart trade fair, which took place from 6 to 8 November

31 October 2018

As the worldwide machine vision market continues to expand – with new trends emerging and new elements coming into play that could impact existing business models – companies are searching for those ever-important opportunities to stimulate growth.

One such trend is almost certainly embedded vision, although the technology behind it is not new, as Mark Williamson, managing director at Stemmer Imaging, noted: ‘Embedded vision is a big topic. However, it has been here a long time, because every smart camera that you buy is an embedded vision system.’

31 October 2018

Qualcomm Technologies' Snapdragon board is designed for mobile devices, but can be used to create other embedded vision systems. Credit: Qualcomm Technologies

Embedded computing promises to lower the cost of building vision solutions, making imaging ubiquitous across many areas of society. Whether this turns out to be the case or not remains to be seen, but in the industrial sector the G3 vision group, led by the European Machine Vision Association (EMVA), is preparing for an influx of embedded vision products with a new standard.