AI for industry

Share this on social media:

Tags: 

Caption: Neural networks can be trained with sample images supplemented with masks of the features to be detected

Michał Czardybon, general manager at Adaptive Vision, gives his view on how to make the most of deep learning in industrial vision

Deep learning is an exciting new technology, but there are still many misconceptions about how it is applied to machine vision applications. There are large neural networks in areas such as autonomous vehicles, customer behaviour analysis or computers playing Go, but here at Adaptive Vision it is used for industrial vision systems, with the biggest focus on quality inspection. In this article I will explain what can really be expected from deep learning for industrial inspection, as well as the system requirements to use it in projects.

Deep learning, in the context of image analysis, can be described as a powerful tool for recognising visual patterns. This might be, for example, a scratch on the surface of a metal part. In the past a sequence of image filtering, blob analysis, feature extraction and rule-based classification tools distinguished the relevant pattern from the background. The problem with this approach is that it requires a lot of engineering time and includes many parameters, which make the algorithms difficult to maintain. Today, things often become too complex to be hand-crafted by engineers; neural networks recognise these patterns in a very different way – by automatically learning from a pool of sample images.

The most typical application of deep learning in industrial vision is recognising if part of an image corresponds to a defect, such as a scratch, or – in a more general sense – to a feature, for example the edge of an object. In such a case the network is trained with sample images supplemented with masks of the features to be detected. It is called supervised training, because the expected outcome is demonstrated explicitly to the network.

Another kind of application is detecting any anomalies in the product under inspection. When it is impossible to define precisely what a defect might be or where it will be located on the image, then semi-unsupervised training is used. In this mode the network is trained on how ‘good’ the objects look. A trained network then analyses an input image and indicates if it looks typical – good – or not. This approach can be used, for example, to verify whether a packaged meal is complete and arranged correctly.

Other notable applications of deep learning tools include general object classification – for example, distinguishing between a chicken leg and a wing – and location, such as finding a part to be picked by a robot.

These are typical applications, but there is still much to say about what is needed in machines or production systems, in order to use neural networks. When talking about industrial inspection, my advice would be to use standard machine vision software, or libraries that include deep learning capabilities. Frameworks like TensorFlow or Caffe are very general and low-level tools that require a good deal of expertise. If one needs traditional template matching or barcode-reading tools, the user could write it, but as standard libraries are effective for all typical cases and extremely well optimised, this is not really justified, neither from a technical, nor an economic point of view.

In terms of hardware, a PC platform with a modern GPU card is necessary to train and run neural networks in a highly parallel way. There is also an option to use a standard PC, but only if the task can afford an image processing time of one second or more – on a GPU it might take 100ms for the same operation.

It is also worth noting that there are some significant differences between standard deep learning tools for industrial applications and the ones used in other markets. First of all, inspection applications usually work with as few as 20 to 50 sample images, provided that their quality is as good as in traditional vision systems. The more images the better, but because of the realities of the industrial vision market, vision engineers tend to rely heavily on image augmentations. These are special transformations that basically create thousands of artificial variations of the real samples.

On the other hand, an important characteristic of industrial deep learning systems is that they still need good lighting, the right lens and an industrial-grade camera to achieve 100 per cent reliability. There are some new application areas where the system can deal with more difficult imaging conditions – especially in automated agriculture – but this does not mean the user can stop thinking about the right hardware setup.

A network can be used to detect undefined defects, such as cracks in a Jaffa Cake

Also, don’t expect deep neural networks to detect defects which a human eye cannot see on the image. Neural networks might be faster, more repetitive and more accurate than humans, assuming limited time per inspection, but they must first be trained with samples manually labelled by us. What is more, the old rule still applies that if there is no information in the image, no algorithm will extract data from it, no matter how good it is.

Finally, I would like to emphasise that deep learning is not a thing of the future, but it is production-ready today. There might be some AI-related buzz in the media, which highlights complexity and concerns about the implications to our lives, but – at least in the field of industrial vision systems – it is simply a new approach to solve an old problem – there is nothing magical.

It is also clear that deep learning will not replace traditional machine vision tools completely. In some applications it is what is needed, but in many others it is only handling part of the task. Moreover, there are some applications, such as measurement or code reading systems, which are best handled with traditional tools.

In general, deep learning is most useful when defects, or other expected outcomes, are defined in an informal way and there is some inherent level of vagueness. Actually, despite wanting to have a 100 per cent well-defined problem, this is rarely the case in visual inspection. Customers and partners often talk about projects using phrases like ‘might be acceptable’ or ‘may vary due to production conditions’. Thus, instead of crafting and tuning complicated sequences of strict traditional tools that need maintenance whenever a tiny detail is changed, deep learning is used that adapts to a specific set of samples it is trained with – and it works with strikingly high reliability.

The whole process also has a big advantage in that it is inherently compliant with test-driven development. Start by clearly defining test cases and any ambiguity must be resolved with the customer up front. All these elements result in a very different way of building inspection systems.

Here, at Adaptive Vision, we strongly believe that neural networks come with so many benefits that they will dominate vision-based quality inspection systems, while at the same time opening a huge number of new applications that could not be solved with traditional tools.

Company: 

Related analysis & opinion

20 November 2018

Deep learning was in force at the Vision trade fair in Stuttgart at the beginning of November. Greg Blackman reports from the show

09 October 2018

A group at the University of Bologna is trying to make images from Grand Theft Auto more realistic so that they can act as training data for neural networks. Greg Blackman listens to Pierluigi Zama Ramirez’s presentation at the European Machine Vision Forum in Bologna in September

19 October 2017

Matthew Dale reports from EMVA’s debut Embedded Vision Europe conference in Stuttgart, where deep learning ‘at the edge’ was discussed

12 September 2017

The machine vision industry is on the verge of entering a new era, one based on embedded processing, according to Basler’s CEO Dr Dietmar Ley. Greg Blackman reports from the European Machine Vision Forum in Vienna

Related features and analysis & opinion

31 October 2018

Machine vision devices are now an increasingly common sight in manufacturing facilities, for inspection and quality control functions across a range of industrial sectors. Some observers are also beginning to explore how companies might make better use of the data they gather from machine vision systems – perhaps by integrating devices into the broader production process, or feeding back the information collected to make ongoing improvements and adjustments to machinery.

31 July 2018

Artificial intelligence (AI) is beginning to infiltrate the machine vision world, just as it is in everyday life. The established industrial vision software firms, such as MVTec Software, Adaptive Vision and Matrox Imaging, are building neural network capabilities into their products, while firms specialising in deep learning and targeting industrial imaging are also springing up.

20 November 2018

Deep learning was in force at the Vision trade fair in Stuttgart at the beginning of November. Greg Blackman reports from the show

16 November 2018

Online retail sales in the US exceeded $453 billion in 2017, according to the US Department of Commerce. Although this may seem like a substantial amount, it only accounts for 13 per cent of the total retail sales made in the region throughout the year, meaning the majority of transactions still take place via the millions of customers walking through their doors and aisles every day.

09 October 2018

A group at the University of Bologna is trying to make images from Grand Theft Auto more realistic so that they can act as training data for neural networks. Greg Blackman listens to Pierluigi Zama Ramirez’s presentation at the European Machine Vision Forum in Bologna in September

31 July 2018

Artificial intelligence (AI) is beginning to infiltrate the machine vision world, just as it is in everyday life. The established industrial vision software firms, such as MVTec Software, Adaptive Vision and Matrox Imaging, are building neural network capabilities into their products, while firms specialising in deep learning and targeting industrial imaging are also springing up.

27 March 2018

A growing number of transport systems around the world are getting smarter as artificial intelligence begins to be deployed. AI algorithms help transport system operators make sense of the data they collect, a big part of which are images.