AI for industry

Share this on social media:

Tags: 

Caption: Neural networks can be trained with sample images supplemented with masks of the features to be detected

Michał Czardybon, general manager at Adaptive Vision, gives his view on how to make the most of deep learning in industrial vision

Deep learning is an exciting new technology, but there are still many misconceptions about how it is applied to machine vision applications. There are large neural networks in areas such as autonomous vehicles, customer behaviour analysis or computers playing Go, but here at Adaptive Vision it is used for industrial vision systems, with the biggest focus on quality inspection. In this article I will explain what can really be expected from deep learning for industrial inspection, as well as the system requirements to use it in projects.

Deep learning, in the context of image analysis, can be described as a powerful tool for recognising visual patterns. This might be, for example, a scratch on the surface of a metal part. In the past a sequence of image filtering, blob analysis, feature extraction and rule-based classification tools distinguished the relevant pattern from the background. The problem with this approach is that it requires a lot of engineering time and includes many parameters, which make the algorithms difficult to maintain. Today, things often become too complex to be hand-crafted by engineers; neural networks recognise these patterns in a very different way – by automatically learning from a pool of sample images.

The most typical application of deep learning in industrial vision is recognising if part of an image corresponds to a defect, such as a scratch, or – in a more general sense – to a feature, for example the edge of an object. In such a case the network is trained with sample images supplemented with masks of the features to be detected. It is called supervised training, because the expected outcome is demonstrated explicitly to the network.

Another kind of application is detecting any anomalies in the product under inspection. When it is impossible to define precisely what a defect might be or where it will be located on the image, then semi-unsupervised training is used. In this mode the network is trained on how ‘good’ the objects look. A trained network then analyses an input image and indicates if it looks typical – good – or not. This approach can be used, for example, to verify whether a packaged meal is complete and arranged correctly.

Other notable applications of deep learning tools include general object classification – for example, distinguishing between a chicken leg and a wing – and location, such as finding a part to be picked by a robot.

These are typical applications, but there is still much to say about what is needed in machines or production systems, in order to use neural networks. When talking about industrial inspection, my advice would be to use standard machine vision software, or libraries that include deep learning capabilities. Frameworks like TensorFlow or Caffe are very general and low-level tools that require a good deal of expertise. If one needs traditional template matching or barcode-reading tools, the user could write it, but as standard libraries are effective for all typical cases and extremely well optimised, this is not really justified, neither from a technical, nor an economic point of view.

In terms of hardware, a PC platform with a modern GPU card is necessary to train and run neural networks in a highly parallel way. There is also an option to use a standard PC, but only if the task can afford an image processing time of one second or more – on a GPU it might take 100ms for the same operation.

It is also worth noting that there are some significant differences between standard deep learning tools for industrial applications and the ones used in other markets. First of all, inspection applications usually work with as few as 20 to 50 sample images, provided that their quality is as good as in traditional vision systems. The more images the better, but because of the realities of the industrial vision market, vision engineers tend to rely heavily on image augmentations. These are special transformations that basically create thousands of artificial variations of the real samples.

On the other hand, an important characteristic of industrial deep learning systems is that they still need good lighting, the right lens and an industrial-grade camera to achieve 100 per cent reliability. There are some new application areas where the system can deal with more difficult imaging conditions – especially in automated agriculture – but this does not mean the user can stop thinking about the right hardware setup.

A network can be used to detect undefined defects, such as cracks in a Jaffa Cake

Also, don’t expect deep neural networks to detect defects which a human eye cannot see on the image. Neural networks might be faster, more repetitive and more accurate than humans, assuming limited time per inspection, but they must first be trained with samples manually labelled by us. What is more, the old rule still applies that if there is no information in the image, no algorithm will extract data from it, no matter how good it is.

Finally, I would like to emphasise that deep learning is not a thing of the future, but it is production-ready today. There might be some AI-related buzz in the media, which highlights complexity and concerns about the implications to our lives, but – at least in the field of industrial vision systems – it is simply a new approach to solve an old problem – there is nothing magical.

It is also clear that deep learning will not replace traditional machine vision tools completely. In some applications it is what is needed, but in many others it is only handling part of the task. Moreover, there are some applications, such as measurement or code reading systems, which are best handled with traditional tools.

In general, deep learning is most useful when defects, or other expected outcomes, are defined in an informal way and there is some inherent level of vagueness. Actually, despite wanting to have a 100 per cent well-defined problem, this is rarely the case in visual inspection. Customers and partners often talk about projects using phrases like ‘might be acceptable’ or ‘may vary due to production conditions’. Thus, instead of crafting and tuning complicated sequences of strict traditional tools that need maintenance whenever a tiny detail is changed, deep learning is used that adapts to a specific set of samples it is trained with – and it works with strikingly high reliability.

The whole process also has a big advantage in that it is inherently compliant with test-driven development. Start by clearly defining test cases and any ambiguity must be resolved with the customer up front. All these elements result in a very different way of building inspection systems.

Here, at Adaptive Vision, we strongly believe that neural networks come with so many benefits that they will dominate vision-based quality inspection systems, while at the same time opening a huge number of new applications that could not be solved with traditional tools.

Company: 

Related analysis & opinion

Credit: Drag&bot

02 July 2020

Christoph Hellmann Santos, of the ROS Industrial Consortium Europe and Fraunhofer IPA, explains why more industrial users are now building robots through the open source software platform, Robot Operating System

18 March 2020

Michał Czardybon, CEO of Adaptive Vision, on how to handle data and image annotation when working with deep learning

27 January 2020

Prior to speaking at the Embedded World trade fair, The Khronos Group’s president, Neil Trevett, discusses the open API standards available for applications using machine learning and embedded vision

13 January 2020

Vassilis Tsagaris and Dimitris Kastaniotis at Irida Labs say an iterative approach is needed to build a real-world AI vision application on embedded hardware

03 December 2019

Following speaking at Embedded Vision Europe, Pierre Gutierrez, lead machine learning researcher at Scortex, writes about the challenges of deploying deep learning on the factory floor

Related features and analysis & opinion

Credit: Drag&bot

02 July 2020

Christoph Hellmann Santos, of the ROS Industrial Consortium Europe and Fraunhofer IPA, explains why more industrial users are now building robots through the open source software platform, Robot Operating System

MVTec’s Halcon software library includes a deep learning OCR tool with pre-trained fonts from a wide range of industries. Credit: MVTec

03 August 2020

Matthew Dale explores vision solutions for code reading and inspection in pharmaceutical production

18 March 2020

Michał Czardybon, CEO of Adaptive Vision, on how to handle data and image annotation when working with deep learning

27 January 2020

Prior to speaking at the Embedded World trade fair, The Khronos Group’s president, Neil Trevett, discusses the open API standards available for applications using machine learning and embedded vision

13 January 2020

Vassilis Tsagaris and Dimitris Kastaniotis at Irida Labs say an iterative approach is needed to build a real-world AI vision application on embedded hardware