AI for industry

Share this on social media:

Tags: 

Caption: Neural networks can be trained with sample images supplemented with masks of the features to be detected

Michał Czardybon, general manager at Adaptive Vision, gives his view on how to make the most of deep learning in industrial vision

Deep learning is an exciting new technology, but there are still many misconceptions about how it is applied to machine vision applications. There are large neural networks in areas such as autonomous vehicles, customer behaviour analysis or computers playing Go, but here at Adaptive Vision it is used for industrial vision systems, with the biggest focus on quality inspection. In this article I will explain what can really be expected from deep learning for industrial inspection, as well as the system requirements to use it in projects.

Deep learning, in the context of image analysis, can be described as a powerful tool for recognising visual patterns. This might be, for example, a scratch on the surface of a metal part. In the past a sequence of image filtering, blob analysis, feature extraction and rule-based classification tools distinguished the relevant pattern from the background. The problem with this approach is that it requires a lot of engineering time and includes many parameters, which make the algorithms difficult to maintain. Today, things often become too complex to be hand-crafted by engineers; neural networks recognise these patterns in a very different way – by automatically learning from a pool of sample images.

The most typical application of deep learning in industrial vision is recognising if part of an image corresponds to a defect, such as a scratch, or – in a more general sense – to a feature, for example the edge of an object. In such a case the network is trained with sample images supplemented with masks of the features to be detected. It is called supervised training, because the expected outcome is demonstrated explicitly to the network.

Another kind of application is detecting any anomalies in the product under inspection. When it is impossible to define precisely what a defect might be or where it will be located on the image, then semi-unsupervised training is used. In this mode the network is trained on how ‘good’ the objects look. A trained network then analyses an input image and indicates if it looks typical – good – or not. This approach can be used, for example, to verify whether a packaged meal is complete and arranged correctly.

Other notable applications of deep learning tools include general object classification – for example, distinguishing between a chicken leg and a wing – and location, such as finding a part to be picked by a robot.

These are typical applications, but there is still much to say about what is needed in machines or production systems, in order to use neural networks. When talking about industrial inspection, my advice would be to use standard machine vision software, or libraries that include deep learning capabilities. Frameworks like TensorFlow or Caffe are very general and low-level tools that require a good deal of expertise. If one needs traditional template matching or barcode-reading tools, the user could write it, but as standard libraries are effective for all typical cases and extremely well optimised, this is not really justified, neither from a technical, nor an economic point of view.

In terms of hardware, a PC platform with a modern GPU card is necessary to train and run neural networks in a highly parallel way. There is also an option to use a standard PC, but only if the task can afford an image processing time of one second or more – on a GPU it might take 100ms for the same operation.

It is also worth noting that there are some significant differences between standard deep learning tools for industrial applications and the ones used in other markets. First of all, inspection applications usually work with as few as 20 to 50 sample images, provided that their quality is as good as in traditional vision systems. The more images the better, but because of the realities of the industrial vision market, vision engineers tend to rely heavily on image augmentations. These are special transformations that basically create thousands of artificial variations of the real samples.

On the other hand, an important characteristic of industrial deep learning systems is that they still need good lighting, the right lens and an industrial-grade camera to achieve 100 per cent reliability. There are some new application areas where the system can deal with more difficult imaging conditions – especially in automated agriculture – but this does not mean the user can stop thinking about the right hardware setup.

A network can be used to detect undefined defects, such as cracks in a Jaffa Cake

Also, don’t expect deep neural networks to detect defects which a human eye cannot see on the image. Neural networks might be faster, more repetitive and more accurate than humans, assuming limited time per inspection, but they must first be trained with samples manually labelled by us. What is more, the old rule still applies that if there is no information in the image, no algorithm will extract data from it, no matter how good it is.

Finally, I would like to emphasise that deep learning is not a thing of the future, but it is production-ready today. There might be some AI-related buzz in the media, which highlights complexity and concerns about the implications to our lives, but – at least in the field of industrial vision systems – it is simply a new approach to solve an old problem – there is nothing magical.

It is also clear that deep learning will not replace traditional machine vision tools completely. In some applications it is what is needed, but in many others it is only handling part of the task. Moreover, there are some applications, such as measurement or code reading systems, which are best handled with traditional tools.

In general, deep learning is most useful when defects, or other expected outcomes, are defined in an informal way and there is some inherent level of vagueness. Actually, despite wanting to have a 100 per cent well-defined problem, this is rarely the case in visual inspection. Customers and partners often talk about projects using phrases like ‘might be acceptable’ or ‘may vary due to production conditions’. Thus, instead of crafting and tuning complicated sequences of strict traditional tools that need maintenance whenever a tiny detail is changed, deep learning is used that adapts to a specific set of samples it is trained with – and it works with strikingly high reliability.

The whole process also has a big advantage in that it is inherently compliant with test-driven development. Start by clearly defining test cases and any ambiguity must be resolved with the customer up front. All these elements result in a very different way of building inspection systems.

Here, at Adaptive Vision, we strongly believe that neural networks come with so many benefits that they will dominate vision-based quality inspection systems, while at the same time opening a huge number of new applications that could not be solved with traditional tools.

Company: 

Related analysis & opinion

Neil Trevett and Chris Yates

16 March 2021

The Khronos Group and the EMVA are to explore software standards for embedded vision. Khronos’ Neil Trevett and EMVA’s Chris Yates explain the work

Credit: Drag&bot

02 July 2020

Christoph Hellmann Santos, of the ROS Industrial Consortium Europe and Fraunhofer IPA, explains why more industrial users are now building robots through the open source software platform, Robot Operating System

05 March 2021

Greg Blackman reports from the Embedded World show, where industry experts gave insights into vision processing at the edge

14 December 2020

Greg Blackman reports on the views of panellists from AIT, MVTec, Irida Labs, and Xilinx discussing AI and machine vision

18 March 2020

Michał Czardybon, CEO of Adaptive Vision, on how to handle data and image annotation when working with deep learning

Related features and analysis & opinion

01 April 2021

Open source software has advanced to a point where it’s now a credible option for industrial imaging, Matthew Dale finds

Neil Trevett and Chris Yates

16 March 2021

The Khronos Group and the EMVA are to explore software standards for embedded vision. Khronos’ Neil Trevett and EMVA’s Chris Yates explain the work

Credit: Drag&bot

02 July 2020

Christoph Hellmann Santos, of the ROS Industrial Consortium Europe and Fraunhofer IPA, explains why more industrial users are now building robots through the open source software platform, Robot Operating System

05 March 2021

Greg Blackman reports from the Embedded World show, where industry experts gave insights into vision processing at the edge

Vision Components MIPI modules can be connected to various embedded processors, including Nvidia Jetson boards

17 February 2021

Greg Blackman examines the effort that goes into creating an embedded vision system

14 December 2020

Greg Blackman reports on the views of panellists from AIT, MVTec, Irida Labs, and Xilinx discussing AI and machine vision

MVTec’s deep learning tool provides means to label data simply and efficiently

10 November 2020

Matthew Dale explores the software to simplify the management and labelling of deep learning data

MVTec’s Halcon software library includes a deep learning OCR tool with pre-trained fonts from a wide range of industries. Credit: MVTec

03 August 2020

Matthew Dale explores vision solutions for code reading and inspection in pharmaceutical production